Riak 1.4 Feature: Protocol Buffers Bucket Properties #256

seancribbs · 2013-06-27T04:09:09Z

Riak 1.4 adds all known, standard bucket properties to the Protocol
Buffers transport for setting and getting, and adds the "reset" or
"clear" feature as well.

Support all bucket properties over PB
Support "reset" or "clear" request/response over PB
Feature detect whether the server supports 1.4 bucket props over PB

Also, some code reorganization was done in codec.py, translate_rw_val was renamed to _encode_quorum.

The bucket fixup for search has been available since 1.0, it is much cleaner and more reliable to use it instead.

seancribbs · 2013-06-27T15:56:50Z

Note that against 1.3.x, I would expect to see these failures:

======================================================================
ERROR: test_disable_search_commit_hook (riak.tests.test_all.RiakPbcTransportTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sean/dev/riak-python-client/riak/tests/test_search.py", line 34, in test_disable_search_commit_hook
    bucket.enable_search()
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 390, in enable_search
    self.set_property('search', True)
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 299, in set_property
    return self.set_properties({key: value})
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 318, in set_properties
    self._client.set_bucket_props(self, props)
  File "/Users/sean/dev/riak-python-client/riak/client/transport.py", line 125, in wrapper
    return self._with_retries(pool, thunk)
  File "/Users/sean/dev/riak-python-client/riak/client/transport.py", line 67, in _with_retries
    return fn(transport)
  File "/Users/sean/dev/riak-python-client/riak/client/transport.py", line 123, in thunk
    return fn(self, transport, *args, **kwargs)
  File "/Users/sean/dev/riak-python-client/riak/client/operations.py", line 90, in set_bucket_props
    return transport.set_bucket_props(bucket, props)
  File "/Users/sean/dev/riak-python-client/riak/transports/pbc/transport.py", line 284, in set_bucket_props
    raise NotImplementedError('Server only supports n_val and '
NotImplementedError: Server only supports n_val and allow_mult properties over PBC

======================================================================
ERROR: test_enable_search_commit_hook (riak.tests.test_all.RiakPbcTransportTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sean/dev/riak-python-client/riak/tests/test_search.py", line 25, in test_enable_search_commit_hook
    bucket.enable_search()
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 390, in enable_search
    self.set_property('search', True)
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 299, in set_property
    return self.set_properties({key: value})
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 318, in set_properties
    self._client.set_bucket_props(self, props)
  File "/Users/sean/dev/riak-python-client/riak/client/transport.py", line 125, in wrapper
    return self._with_retries(pool, thunk)
  File "/Users/sean/dev/riak-python-client/riak/client/transport.py", line 67, in _with_retries
    return fn(transport)
  File "/Users/sean/dev/riak-python-client/riak/client/transport.py", line 123, in thunk
    return fn(self, transport, *args, **kwargs)
  File "/Users/sean/dev/riak-python-client/riak/client/operations.py", line 90, in set_bucket_props
    return transport.set_bucket_props(bucket, props)
  File "/Users/sean/dev/riak-python-client/riak/transports/pbc/transport.py", line 284, in set_bucket_props
    raise NotImplementedError('Server only supports n_val and '
NotImplementedError: Server only supports n_val and allow_mult properties over PBC

======================================================================
ERROR: test_primary_quora (riak.tests.test_all.RiakPbcTransportTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sean/dev/riak-python-client/riak/tests/test_kv.py", line 471, in test_primary_quora
    self.assertEqual(bucket.pr, 0)
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 30, in _prop_getter
    return self.get_property(name)
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 309, in get_property
    return self.get_properties()[key]
KeyError: 'pr'

======================================================================
ERROR: test_rw_settings (riak.tests.test_all.RiakPbcTransportTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sean/dev/riak-python-client/riak/tests/test_kv.py", line 446, in test_rw_settings
    self.assertEqual(bucket.r, "quorum")
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 30, in _prop_getter
    return self.get_property(name)
  File "/Users/sean/dev/riak-python-client/riak/bucket.py", line 309, in get_property
    return self.get_properties()[key]
KeyError: 'r'

======================================================================
FAIL: test_clear_bucket_properties (riak.tests.test_all.RiakPbcTransportTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sean/dev/riak-python-client/riak/tests/test_kv.py", line 491, in test_clear_bucket_properties
    self.assertTrue(bucket.clear_properties())
AssertionError: False is not true

----------------------------------------------------------------------

evanmcc · 2013-06-29T17:33:36Z

Read over the code on friday, looks good, as noted, some nice use of metaprogramming going on there. Going to do a final pass and some testing on monday.

hazen · 2013-06-30T17:34:43Z

Verified Sean's errors against 1.3.1:

ERROR: test_disable_search_commit_hook (riak.tests.test_all.RiakPbcTransportTestCase)
NotImplementedError: Server only supports n_val and allow_mult properties over PBC
======================================================================
ERROR: test_enable_search_commit_hook (riak.tests.test_all.RiakPbcTransportTestCase)
NotImplementedError: Server only supports n_val and allow_mult properties over PBC
======================================================================
ERROR: test_primary_quora (riak.tests.test_all.RiakPbcTransportTestCase)
KeyError: 'pr'
======================================================================
ERROR: test_rw_settings (riak.tests.test_all.RiakPbcTransportTestCase)
KeyError: 'r'
======================================================================
FAIL: test_clear_bucket_properties (riak.tests.test_all.RiakPbcTransportTestCase)
AssertionError: False is not true

hazen · 2013-06-30T20:54:18Z

Tested against 1.4.0rc1 and it looks good:

----------------------------------------------------------------------
Ran 208 tests in 31.445s

OK

seancribbs · 2013-07-01T13:14:44Z

Thanks @javajolt, I'll merge late this morning unless @evanmcc has objections.

Riak 1.4 Feature: Protocol Buffers Bucket Properties

ghost assigned seancribbs Jun 26, 2013

Sean Cribbs added 9 commits June 26, 2013 09:31

Bump client version and riak_pb dep for Riak 1.4 features.

61a05b4

Unify the bucket properties tests since PBC will support all properties.

5f4b588

Add support for all bucket properties and clearing in PB.

cdcab15

Also, some code reorganization was done in codec.py, translate_rw_val was renamed to _encode_quorum.

Add some much needed documentation for other PB codec methods.

43b663e

Cleanup bucket properties shortcuts with a helper function.

caf7faf

Use the search property instead of directly modifying precommit.

d94a16f

The bucket fixup for search has been available since 1.0, it is much cleaner and more reliable to use it instead.

Fix a few doc string problems in bucket.py.

a26a699

Predicate the bucket properties features on version detection.

cc54a05

Apply some PEP8 and pyflakes fixes.

a31efe1

seancribbs pushed a commit that referenced this pull request Jul 1, 2013

Merge pull request #256 from basho/gh256-1.4-bucket-props

3c5df47

Riak 1.4 Feature: Protocol Buffers Bucket Properties

seancribbs merged commit 3c5df47 into master Jul 1, 2013

seancribbs deleted the gh256-1.4-bucket-props branch July 1, 2013 16:32

seancribbs removed their assignment May 8, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Riak 1.4 Feature: Protocol Buffers Bucket Properties #256

Riak 1.4 Feature: Protocol Buffers Bucket Properties #256

Uh oh!

seancribbs commented Jun 27, 2013

Uh oh!

seancribbs commented Jun 27, 2013

Uh oh!

evanmcc commented Jun 29, 2013

Uh oh!

hazen commented Jun 30, 2013

Uh oh!

hazen commented Jun 30, 2013

Uh oh!

seancribbs commented Jul 1, 2013

Uh oh!

Uh oh!

Riak 1.4 Feature: Protocol Buffers Bucket Properties #256

Riak 1.4 Feature: Protocol Buffers Bucket Properties #256

Uh oh!

Conversation

seancribbs commented Jun 27, 2013

Uh oh!

seancribbs commented Jun 27, 2013

Uh oh!

evanmcc commented Jun 29, 2013

Uh oh!

hazen commented Jun 30, 2013

Uh oh!

hazen commented Jun 30, 2013

Uh oh!

seancribbs commented Jul 1, 2013

Uh oh!

Uh oh!