-
Notifications
You must be signed in to change notification settings - Fork 402
Comparing changes
Open a pull request
base repository: hypermodeinc/ristretto
base: v0.1.1
head repository: hypermodeinc/ristretto
compare: v0.2.0
- 17 commits
- 49 files changed
- 12 contributors
Commits on Dec 8, 2022
-
chore(ci): pin runner to ubuntu 20.04 (#319)
## Problem Lint tests and CI tests were failing. ## Solution Pin ristretto test runner to Ubuntu 20.04, and bump up linter dependencies.
Configuration menu - View commit details
-
Copy full SHA for 679ebab - Browse repository at this point
Copy the full SHA 679ebabView commit details
Commits on Mar 2, 2023
-
chore(ci): resolve community PR failures (#330)
Goveralls was failing on community PR's. See hypermodeinc/badger#1894 Resolves: coverallsapp/github-action#150
Configuration menu - View commit details
-
Copy full SHA for f4fa6df - Browse repository at this point
Copy the full SHA f4fa6dfView commit details
Commits on Mar 21, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 3177d9c - Browse repository at this point
Copy the full SHA 3177d9cView commit details
Commits on Apr 19, 2023
-
chore(ci): add security scanning workflow (#338)
Add trivy security scanning workflow to ristretto.
Configuration menu - View commit details
-
Copy full SHA for 34fd2d3 - Browse repository at this point
Copy the full SHA 34fd2d3View commit details
Commits on May 16, 2023
-
docs: format example on readme (#339)
The previous examples were missing package references. Now change to a complete example. If you want to test this library, just run it directly.
Configuration menu - View commit details
-
Copy full SHA for ebc9d9e - Browse repository at this point
Copy the full SHA ebc9d9eView commit details -
chore(tests): fix flaky test TestDropUpdates (#334)
While submitting a documentation PR, CI failed to run my change with the following flaky error: cache_test.go:866: Error Trace: cache_test.go:866 cache_test.go:872 Error: Should be true Test: TestDropUpdates This happens because sometimes sleeping for 1 ms is not enough to ensure the background processing happens. Replace with c.Wait(), which ensures it waits until everything is processed. There is a second race, with the full output below. It is caused when c.Close() is called with the last Set() still in the buffer. This causes onEvict() to be called for every item. Call c.Wait() before that point also. I was able to reproduce these flakes using my "makeflaky" tool: https://github.com/evanj/hacks#makeflaky-pause-a-process-to-make-tests-flaky Running it with the following arguments: makeflaky -goTest . -race -v -run=TestDropUpdates -count=1000 -failfast
Configuration menu - View commit details
-
Copy full SHA for c476529 - Browse repository at this point
Copy the full SHA c476529View commit details -
docs: document Wait, clarify Get (#333)
* Document the Wait() function * Clarify that Get() will never return expired items * Clarify in what circumstances Get() can return (nil, true)
Configuration menu - View commit details
-
Copy full SHA for 9577449 - Browse repository at this point
Copy the full SHA 9577449View commit details
Commits on Jul 2, 2023
-
Configuration menu - View commit details
-
Copy full SHA for c73d585 - Browse repository at this point
Copy the full SHA c73d585View commit details
Commits on Aug 29, 2023
-
fix: support compilation to wasip1 (#344)
## Problem I'm interested in getting ristretto to build to the new `wasip1` target, but it fails due to ristretto's assumption that the program is being compiled to a unix system where files can be mapped to memory. At this time, the `x/sys/unix` package does not consider `wasip1` to be a unix platform, and memory mapping isn't supported either when compiling to WebAssembly. ``` $ GOOS=wasip1 GOARCH=wasm gotip build package github.com/dgraph-io/ristretto imports github.com/dgraph-io/ristretto/z imports golang.org/x/sys/unix: build constraints exclude all Go files in /go/pkg/mod/golang.org/x/[email protected]/unix ``` ## Solution The proposed solution in this PR is to provide stubs implementations of the memory mapping functions which return `ENOSYS` to indicate that the functionalities are not available when compiled to `GOOS=wasip1`. If I'm reading the code correctly, this means that persisting data structures to a storage medium will not be supported, but the rest of ristretto should work as expected. I believe this to be a useful trade-off since many dependents of ristretto only use the in-memory data structures (e.g. https://github.com/DataDog/datadog-agent/blob/main/pkg/obfuscate/cache.go, which is what I'm interested in getting to compile). A more advanced solution could be to rework the `z.Buffer` internals so the data structure can work on platforms where mapping files to memory isn't supported; however, this represents a bigger undertaking, I would like to first unblock dependencies by allowing the program to compile. Signed-off-by: Achille Roussel <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 3f6b44a - Browse repository at this point
Copy the full SHA 3f6b44aView commit details
Commits on Aug 30, 2023
-
glog adds global flags so it is not a good library (adding to an existing program causes a panic if a flag name conflicts)
Configuration menu - View commit details
-
Copy full SHA for bdcf5e9 - Browse repository at this point
Copy the full SHA bdcf5e9View commit details
Commits on Aug 31, 2023
-
Configuration menu - View commit details
-
Copy full SHA for c5789d6 - Browse repository at this point
Copy the full SHA c5789d6View commit details -
add config for cleanup ticker duration (#342)
## Problem Cleanup ticker to remove entries for expired keys taking a lot of CPU cycles around 90% in the case of pod with 1.6 CPU ## Solution Making the duration of the cleanup ticker configurable so users can configure it to higher values and can save on CPU cycles.
Configuration menu - View commit details
-
Copy full SHA for e8dc5b0 - Browse repository at this point
Copy the full SHA e8dc5b0View commit details -
set missing Expiration field on evicted items (#345)
On our use case we need to rely on expiration timestamps but we realized that they return zero timestamps. Our assumption is that this is due to a bug and not by design. ## Problem When `OnEvict` callback is executed `Item` does not have `Expiration` field set. Considering it is a public field this needs to be set and available. ## Solution Initialize `Expiration` field before calling the `onEvict` callback
Configuration menu - View commit details
-
Copy full SHA for f0e7027 - Browse repository at this point
Copy the full SHA f0e7027View commit details -
chore(ci): separate out coverage report workflow (#353)
* separate ristretto test and coverage workflows
Configuration menu - View commit details
-
Copy full SHA for e6d62cb - Browse repository at this point
Copy the full SHA e6d62cbView commit details
Commits on Sep 1, 2023
-
uint32 to uint64 in slice methods (#323)
## Problem I'm currently on Dgraph 21.12 unable to export my data. Export fails on just two nodes out of 7 with the "Unexpected EOF" error: ``` Dec 14 11:31:51 dm-dgraph-04 dgraph[224355]: I1214 11:31:51.840130 224355 log.go:34] Export [01h26m22s] Scan (12): ~2.1 TiB/2.5 TiB at 177 MiB/sec. Sent: 801.5 GiB at 231 MiB/sec. jemalloc: 7.5 GiB Dec 14 11:31:55 dm-dgraph-04 dgraph[224355]: W1214 11:31:55.408201 224355 log.go:36] Error while sending: unexpected EOF ``` Skipping rather long investigation of this issue I came to find length of slice, written to the Buffer during export exceed the size of uint32 (i've decoded varint before Value field in Badger KV struct with RDF's to get something around 4.5Gb, which is expected for a rather bloated reverse edge to the one of the most common nodes in my DB. Also count query returns 72 105 794 connected nodes which is, welp, quite a lot). Not to mention that working with `int` which is almost always is `int64` and then casually casting it to `uint32` w/o any checks or warnings is as bad as it gets. ## Solution Find any `4` and `Uint32` and carefully replace them with `8` and `Uint64`. As this happens only in slice-related methods the fix is quite easy. Locally tests run just fine, but i had to patch the sort one to accommodate for size changes. Also i did test 21.12-related badger version and tests run fine too.
Configuration menu - View commit details
-
Copy full SHA for c00b352 - Browse repository at this point
Copy the full SHA c00b352View commit details -
stop cleanupTicker while closing cache (#343)
## Problem `cleanupTicker `that is created in `NewCache` is never stopped, which is why the code below causes a memory leak ```go for { c, _ := NewCache(...) c.Close() } ``` ## Solution Stop `cleanupTicker` in `Close` method
Configuration menu - View commit details
-
Copy full SHA for 9144662 - Browse repository at this point
Copy the full SHA 9144662View commit details
Commits on Sep 4, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 1854617 - Browse repository at this point
Copy the full SHA 1854617View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff v0.1.1...v0.2.0