Skip to content

Commit 2a4b4e9

Browse files
authored
Merge pull request JanusGraph#875 from chupman/whirr-cleanup
Removed whirr specific files and ec2 hbase deploy doc using whirr [skip ci]
2 parents 4ccf5a4 + f225304 commit 2a4b4e9

File tree

3 files changed

+0
-142
lines changed

3 files changed

+0
-142
lines changed

docs/hbase.adoc

Lines changed: 0 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -86,85 +86,6 @@ Please refer to the http://hbase.apache.org/book/config.files.html[HBase configu
8686

8787
JanusGraph over HBase supports global vertex and edge iteration. However, note that all these vertices and/or edges will be loaded into memory which can cause `OutOfMemoryException`. Use <<hadoop-tp3>> to iterate over all vertices or edges in large graphs effectively.
8888

89-
=== Deploying on Amazon EC2
90-
91-
[quote, 'http://aws.amazon.com/ec2/[Amazon EC2]']
92-
Amazon EC2 is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.
93-
94-
Follow these steps to setup an HBase cluster on EC2 and deploy JanusGraph over HBase. To follow these instructions, you need an Amazon AWS account with established authentication credentials and some basic knowledge of AWS and EC2.
95-
96-
The following commands first launch a four-node HBase cluster on EC2 via http://whirr.apache.org/[Whirr], then run a basic JanusGraph test case using the cluster.
97-
98-
The configuration described below puts one HBase master server in charge of three HBase regionservers. The master will be the sole member of the Zookeeper quorum by which JanusGraph connects to HBase.
99-
100-
Whirr 0.7.1 sometimes fails when run on a machine behind a NAT https://issues.apache.org/jira/browse/WHIRR-459[WHIRR-459]. For this reason, it's recommended to use at least Whirr 0.7.2. Whirr 0.8.0 was used to test the following commands on a t1.micro instance running Amazon Linux 2012.03. These commands might need tweaking to produce the intended results on environments besides a t1.micro instance running Amazon Linux 2012.03.
101-
102-
[source, bourne]
103-
----
104-
# These commands were executed on a t1.micro instance running Amazon Linux 2012.03 x86_64.
105-
# The AMI identifier for Amazon Linux 2012.03 x86_64 is ami-aecd60c7.
106-
# https://console.aws.amazon.com/ec2/home?region=us-east-1#launchAmi=ami-aecd60c7
107-
export AWS_ACCESS_KEY_ID=... # Set your Access Key here
108-
export AWS_SECRET_ACCESS_KEY=... # Set your Secret Key here
109-
curl -O http://www.apache.org/dist/whirr/whirr-0.8.0/whirr-0.8.0.tar.gz
110-
tar -xzf whirr-0.8.0.tar.gz && cd whirr-0.8.0
111-
# Generate an SSH keypair with which Whirr will deploy and manage instances
112-
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa_whirr
113-
# Download a Whirr recipe for deploying HBase 0.94.1 with hadoop-core 1.0.3
114-
pushd recipes && wget 'https://raw.github.com/JanusGraph/janusgraph/master/config/whirr-hbase.properties' ; popd
115-
bin/whirr launch-cluster --config recipes/whirr-hbase.properties --private-key-file ~/.ssh/id_rsa_whirr
116-
# Run a superficial health check on the hbase-master node (this should print "imok")
117-
echo "ruok" | nc $(awk '{print $3}' ~/.whirr/hbase-testing/instances | head -1) 2181; echo
118-
# Login to the HBase master node to run the remaining commands
119-
ssh -i ~/.ssh/id_rsa_whirr -o "UserKnownHostsFile /dev/null" \
120-
-o StrictHostKeyChecking=no \
121-
`grep hbase-master ~/.whirr/hbase-testing/instances \
122-
| awk '{print $3}'`
123-
# Maven 2 is available through the package manager, but an incompatibility
124-
# with surefire 2.12 makes it a pain to use; here we download Maven 3 without
125-
# the OS package manager
126-
wget 'http://archive.apache.org/dist/maven/maven-3/3.0.4/binaries/apache-maven-3.0.4-bin.tar.gz'
127-
tar -xzf apache-maven-3.0.4-bin.tar.gz
128-
# Install git
129-
sudo apt-get install -y git-core
130-
# Clone JanusGraph
131-
git clone 'git://github.com/JanusGraph/janusgraph.git' && cd janusgraph
132-
# Run a HBase-backed test of JanusGraph
133-
#
134-
# This test should produce pages of output ending in something like this:
135-
#
136-
# -------------------------------------------------------
137-
# T E S T S
138-
# -------------------------------------------------------
139-
# Running org.janusgraph.graphdb.hbase.ExternalHBaseGraphPerformanceTest
140-
# Starting trial 1/1
141-
# 10000
142-
# 20000
143-
# 30000
144-
# 40000
145-
# 50000
146-
# Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 303.659 sec
147-
#
148-
# Results :
149-
#
150-
# Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
151-
#
152-
# [INFO] ------------------------------------------------------------------------
153-
# [INFO] BUILD SUCCESS
154-
# [INFO] ------------------------------------------------------------------------
155-
~/apache-maven-3.0.4/bin/mvn test -Dtest=ExternalHBaseGraphPerformanceTest#unlabeledEdgeInsertion
156-
# Check on hadoop
157-
hadoop version # Should print 1.0.3
158-
# List the hadoop root; should print something like:
159-
#
160-
# Found 4 items
161-
# drwxr-xr-x - hadoop supergroup 0 2012-09-20 00:20 /hadoop
162-
# drwxr-xr-x - hadoop supergroup 0 2012-09-20 00:42 /hbase
163-
# drwxrwxrwx - hadoop supergroup 0 2012-09-20 00:20 /tmp
164-
# drwxrwxrwx - hadoop supergroup 0 2012-09-20 00:20 /user
165-
hadoop fs -ls /
166-
----
167-
16889
=== Tips and Tricks for Managing an HBase Cluster
16990

17091
The http://wiki.apache.org/hadoop/Hbase/Shell[HBase shell] on the master server can be used to get an overall status check of the cluster.

janusgraph-cassandra/config/whirr-cassandra.properties

Lines changed: 0 additions & 28 deletions
This file was deleted.

janusgraph-hbase-parent/common/test/conf/whirr-hbase.properties

Lines changed: 0 additions & 35 deletions
This file was deleted.

0 commit comments

Comments
 (0)