Skip to content

[DON'T MERGE] prototype new uncompatible Snark Worker rework #17179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 165 commits into
base: compatible
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
165 commits
Select commit Hold shift + click to select a range
8596f3e
Expose of_yojson for txn witness, implement a write_all_proofs_to_dis…
glyh Apr 24, 2025
002e4e0
implement `drop_sok` for converting Snarked_ledger_state.With_sok to …
glyh Apr 24, 2025
8b82e91
Implement Snark_work_lib.Partitioned, this would later track works di…
glyh Apr 24, 2025
875526a
(Snark Work Lib) Rename all UUID to ID, because we recycled them and …
glyh Apr 25, 2025
ba757fc
(Snark Work Lib) Rename Zkapp_command_job.pairing_id -> pairing
glyh Apr 25, 2025
f398a03
Merge branch 'corvo/gather-selector-work-definition' into corvo/defin…
glyh Apr 25, 2025
6783088
(Snark Work Lib) rename `materialize` -> `read_all_proofs_from_disk`;…
glyh Apr 25, 2025
00b207e
(Snark Work Lib) Redesign Pairing so we're using less complicated pai…
glyh Apr 25, 2025
dd38618
(Snark Work Lib) Redesign Partitioned.Spec so there's no invalid state
glyh Apr 25, 2025
958e035
(Snark Work Lib) Lift a Poly version of Partitioned.Spec
glyh Apr 25, 2025
d217b78
(Snark Work Lib) redesign Partitioned.Result
glyh Apr 25, 2025
266db5d
(Snark Work Lib) rename Partitioned.Spec.{Regular -> Single}
glyh Apr 25, 2025
bc5c91d
Merge branch 'corvo/gather-selector-work-definition' into corvo/defin…
glyh Apr 27, 2025
328024b
Add ~proof_cache_db to some auxiliary functions, so we pass compilati…
glyh Apr 27, 2025
edadc19
snark_work_lib/dune: run dune fmt --auto-promote
dannywillems Apr 28, 2025
dcfdd0c
snark_work_lib/dune: reorder alphabetically
dannywillems Apr 28, 2025
244319e
snark_work_lib/dune: remove unused deps mina_state
dannywillems Apr 28, 2025
d954607
Snark Work Lib: Implement auxilary function Selector.Result.transactions
glyh Apr 29, 2025
e9eb39f
Snark Work Lib: track fees in all branches of a Partitioned.Spec so w…
glyh Apr 29, 2025
25eb27e
Snark Work Lib: Implement Partitioned.Spec.Poly.fee_of_full
glyh Apr 29, 2025
451ea49
Snark Work Lib: Split Partitioned.Pairing.t to Partitioned.Pairing.{S…
glyh Apr 30, 2025
92676cf
Snark Work Lib: Factor out construct_selector_result from Partitioned…
glyh Apr 29, 2025
0e0b136
Snark Work Lib: Implement Partitioned.Result.to_spec
glyh Apr 29, 2025
c39b3a2
Snark Work Lib: Implement Partitioned.Zkapp_command_job.Spec.statemen…
glyh Apr 29, 2025
435ce8a
Mina Stdlib: Add Time.Span that has yojson & sexp implementation
glyh Apr 30, 2025
15bfae5
Snark Worker: erase old interface, create a brand new interface that …
glyh Apr 30, 2025
eea70b6
Snark Work Lib: Refactor so we're always tracking issue time
glyh Apr 30, 2025
16246c1
Snark Worker: rewrite debug.ml to conform new interface
glyh Apr 30, 2025
bc0e1ca
Snark Work Lib: rename Partitioned.{SpecCommon -> Spec_common}
glyh Apr 30, 2025
5a1b025
Snark Worker: merge {m, proof_level} in Worker_state to together, so …
glyh Apr 30, 2025
d910c17
Snark Work Lib: Implement transactions helper for stable selector specs
glyh Apr 30, 2025
be761b9
Snark Worker: Simplify prod worker, lift a `perform` function so we c…
glyh Apr 30, 2025
3e1f4bf
Snark Worker: replace Snark_worker.Prod.single_spec -> Work.Selector.…
glyh Apr 30, 2025
83b8e9d
Snark Worker: In Prod worker, lift log_base_snark to top level as log…
glyh Apr 30, 2025
97a0cec
Snark Worker: In Prod worker, lift log_merge_snark to top level as lo…
glyh Apr 30, 2025
b7d5fd2
Snark Worker: Lift `cache_and_time` that deals with caching and timin…
glyh Apr 30, 2025
a8bed0d
Snark Worker: Lift closure inside perform_single to perform_single_ra…
glyh Apr 30, 2025
1c64430
Snark Worker: put all structure log events in module Events
glyh Apr 30, 2025
152786f
Snark Worker: put Get_work RPC in Rpc_get_work
glyh Apr 30, 2025
68c6b3a
Snark Worker: put Submit_work RPC in Rpc_submit_work
glyh Apr 30, 2025
d0e1bd9
Snark Worker: put Failed_to_generate_snark RPC in Rpc_failed_to_gener…
glyh Apr 30, 2025
7430ec8
Snark Worker: remove Rpcs, Snark_worker, port relevant comments to Rp…
glyh Apr 30, 2025
23d453f
Snark Worker: Rename Functor -> Entry
glyh Apr 30, 2025
3a94c0d
Snark Worker: add back previously deleted `command_name` in entry
glyh Apr 30, 2025
7d07f45
Snark Worker: drop Functor in entry
glyh Apr 30, 2025
77cfcc4
Snark Worker: rewrite a Snark_worker interface
glyh Apr 30, 2025
b9aeca7
Uptime Service: port Snark worker to adapt to snark worker refactor
glyh Apr 30, 2025
1aa7da6
Snark Work Debugger: port to use new Snark Worker API
glyh Apr 30, 2025
056ee30
Mina Lib: port to adapt Snark Worker new API
glyh Apr 30, 2025
2316231
Run Snark Worker: port to adapt new Snark Worker API
glyh Apr 30, 2025
e4a0b49
Integration Test Lib: Port to adapt to new Snark Worker API
glyh Apr 30, 2025
c4eb3c8
Init.Mina_run: Port to adapt new Snark Worker API
glyh Apr 30, 2025
2d90d4a
Mina Cli Entrypoint: port to adapt new Snark Worker API
glyh Apr 30, 2025
8f2ea66
Snark Work Lib: document Partitioned
glyh May 1, 2025
c3bb961
FIX(Mina Stdlib): Use a more canonical representation of time span so…
glyh May 1, 2025
e6baf12
FIX(Snark Work Lib): Partitioned.Result.of_selector_result should acc…
glyh May 1, 2025
51196b1
Snark Work Lib: make `issued_since_unix_epoch` mandatory for conversi…
glyh May 1, 2025
dc96883
Snark Work Lib: rename Partitioned.Spec.Poly.{map_metric -> map, map_…
glyh May 1, 2025
707fdf4
FIX(Snark Worker): drop all OPAM Libraries/Local Libraries comments
glyh May 1, 2025
272b945
Snark Worker: Conform hyprlinks to odoc convention
glyh May 1, 2025
99f4bc4
FIX(Snark Worker): use stable sok digest in argument of perform
glyh May 1, 2025
a825be9
Snark Worker: create interface for Snark_worker, move all comments {…
glyh May 1, 2025
5001d62
Merge branch 'corvo/define-partitioned-work' into corvo/refactor-snar…
glyh May 1, 2025
918e5cd
Snark Worker: remove redundant module alias
glyh May 1, 2025
b6638a5
FIX(Uptime Snark Worker): return an Error instead of failwith
glyh May 1, 2025
4a6f607
Snark Worker: Rewrite Worker interface's document
glyh May 1, 2025
7c1b132
Snarked_Ledger_state: remove auxilary definition `regular_snarked_led…
glyh May 3, 2025
a0a05e9
Mina_stdlib: versioning type `Time.Span` should start from V1
glyh May 3, 2025
c53fa96
Merge branch 'compatible' into corvo/define-partitioned-work
glyh May 3, 2025
502bb13
Mina Wire Types: Expose the fact that a transaction_snark is a proof …
glyh May 3, 2025
58875f5
Ledger Proof: refactor out a Poly version of Ledger_proof that could …
glyh May 3, 2025
d3bdf81
Snark Work Lib: Lift a Poly version of Zkapp_command_job.Spec
glyh May 3, 2025
412d511
Transaction Witness: refactor out a Poly version of Transaction Witne…
glyh May 3, 2025
0ea9d43
Snark Work Lib: Polymorphize Selector.Single.Spec.transaction to Work…
glyh May 3, 2025
a9df6d2
Snark Work Lib: Polymorphize Partitioned.Zkapp_command_job
glyh May 3, 2025
954f6ed
Snark Work Lib: Polymorphize Partitioned.Spec
glyh May 3, 2025
82d1045
Snark Work Lib: Polymorphize Partitioned.{Proof_with_metric, Result}
glyh May 3, 2025
ce05137
Snark Work Lib: Expose to_spec, of_selector_result, to_selector_resul…
glyh May 3, 2025
9fd4a05
Snark Work Lib: relax types for Partitioned.construct_selector_result
glyh May 3, 2025
2fc1550
FIX(Snark Work Lib): fix definitions & helper funcitons in Partitione…
glyh May 3, 2025
21b678d
Merge corvo/define-partitioned-work and resolve conflicts
glyh May 6, 2025
dba44c6
Work_partitioner: set up library
glyh Apr 19, 2025
06693b6
(Work Partitioner) Add module `Id_generator` to `Work_partitioner`
glyh Apr 19, 2025
139b539
(Work Partitioner) Add functor Job_pool
glyh Apr 19, 2025
a75bd0f
Snark Worker, Work Partitioner: Factor out common function extract_zk…
glyh Apr 30, 2025
f89b177
Work Partitioner: add Work_partitioner.Pending_zkapp_command
glyh Apr 30, 2025
f116bdb
Work Partitioner: Add {Zkapp_command_job_pool, Sent_job_pool}
glyh Apr 30, 2025
54a7b94
(Work Partitioner) Add Mergable_single_work
glyh Apr 25, 2025
9f530d0
Work Partitioner: add Work_partitioner.t and corresponding initializer
glyh Apr 30, 2025
ae7c6e6
Work Partitioner: Add Work_partitioner.reissue_old_task
glyh Apr 25, 2025
1952c76
Work Partitioner: Add Work_partitioner.issue_from_zkapp_command_work_…
glyh Apr 30, 2025
d976deb
Work Partitioner: Add Work_partitioner.Work_partitioner.{issue_from_t…
glyh Apr 30, 2025
87c7314
Work Partitioner: Add function Work_partitioner.consume_job_from_sel…
glyh Apr 25, 2025
51920ed
Work Partitioner: Implement {request_from_selector_and_consume_by_pa…
glyh Apr 30, 2025
77d72e9
Work Partitioner: add Work_partitioner.submit_result
glyh Apr 30, 2025
f49b4d4
Work Partitioner: implement submit_single
glyh Apr 26, 2025
cd50d63
Work Partitioner: implement submit_into_pending_zkapp_command
glyh Apr 30, 2025
9d26f6b
Work Partitioner: implement submit_partitioned_work
glyh Apr 26, 2025
7bfb930
Work Partitioner: Refactor Job_pool.{find_first_ready -> fold_until}
glyh May 1, 2025
8c501fe
FIX(Work Partitioner): go through all pending zkapp command to issue …
glyh May 1, 2025
daa37a2
FIX(Work Partitioner): we should prioritize completion of work on han…
glyh May 1, 2025
4d11b63
Snark Worker: update Get_work master RPC
glyh May 3, 2025
1493013
Snark Worker: Implement Get_work RPC V3
glyh May 3, 2025
fab9ad4
Snark Worker: update Submit_work master RPC
glyh May 3, 2025
d47ce10
Snark Worker: Implement Submit_work RPC V3
glyh May 3, 2025
fe2c9b5
Snark Worker: update Failed_to_generate_snark master RPC
glyh May 3, 2025
98ebb33
Snark Worker: Implement Failed_to_generate_snark RPC V3
glyh May 3, 2025
559f878
Snark Worker, Work Partitioner: Lift definition of Zkapp_command_inpu…
glyh May 3, 2025
ab0ca9a
Snark Worker: update Implementation interface
glyh May 3, 2025
8710742
Snark Worker: port Production worker to new interface
glyh May 3, 2025
e279538
Mina Run, Snark Worker, Snark Work Lib: Factor out common metric func…
glyh May 5, 2025
a773cf5
Snark Work Lib: expose Spec.Poly.fee_of_full
glyh May 3, 2025
e8123dc
Snark Worker: port Entry to adapt to new RPC
glyh May 5, 2025
3d0ef8e
Uptime Service: port Uptime Snark worker to adapt to new Snark Worker…
glyh May 5, 2025
22a8005
Standalone Snark Worker: port to use new Snark Worker RPC
glyh May 5, 2025
acb0ef1
Mina Lib: add work partitioner to snark_jobs_state
glyh May 5, 2025
b69360f
Mina Lib: update request_work function to utilize Work Partitioner
glyh May 5, 2025
1d04689
Mina Lib: update add_work function to utilize Work Partitioner
glyh May 5, 2025
c8b7852
Mina Run: Port to adapt new Snark Worker RPC
glyh May 5, 2025
fafb971
Mina Cli Entrypoint: port to adapt new Snark Worker RPC
glyh May 5, 2025
4027805
PR: add CHANGELOG
glyh Apr 27, 2025
a348a99
Work Partitioner: remove recycle mechanism in ID generator and use a …
glyh May 12, 2025
866ae90
Work Partitioner: accept timeouted SNARK worker to submit a proof
glyh May 12, 2025
ce5d478
Merge branch 'corvo/implement-work-partitioner-3' into corvo/update-s…
glyh May 12, 2025
07d642d
remove old RPC
glyh May 13, 2025
933ed6a
WIP
glyh May 13, 2025
9399cc0
WIP
glyh May 13, 2025
5d6e3a2
WIP
glyh May 14, 2025
d5461c9
wip
glyh May 14, 2025
220d7a7
wip
glyh May 14, 2025
de4b2e1
WIP
glyh May 14, 2025
f6fd073
WIP
glyh May 14, 2025
e92045e
WIP
glyh May 14, 2025
6f63d32
WIP
glyh May 14, 2025
1587fe1
WIP
glyh May 15, 2025
29153b5
WIP
glyh May 15, 2025
1b91d9d
WIP
glyh May 15, 2025
c91a0f4
WIP
glyh May 15, 2025
32da50b
WIP
glyh May 15, 2025
9a66810
WIP and nuked work selector reissuing
glyh May 15, 2025
b2d7885
remove transaction snark Poly
glyh May 15, 2025
014c92d
WIP
glyh May 15, 2025
87196c7
wip
glyh May 15, 2025
ddc20ea
WIP
glyh May 16, 2025
a96c3bd
WIP
glyh May 16, 2025
6bf1c25
WIP
glyh May 16, 2025
11c1ed9
WIP
glyh May 16, 2025
821fdf9
TODO
glyh May 16, 2025
08f9a4b
WIP
glyh May 16, 2025
673f8a2
WIP
glyh May 16, 2025
910f27a
1
glyh May 16, 2025
82b34c7
1
glyh May 16, 2025
c1577d4
a
glyh May 16, 2025
12f1dd1
WIP
glyh May 16, 2025
9a29ca3
upd
glyh May 17, 2025
4ba8b37
a
glyh May 24, 2025
4103384
wip
glyh May 27, 2025
149ceae
WIP
glyh May 27, 2025
c52f377
don't store spec in result
glyh May 27, 2025
ab85dd0
wip
glyh May 27, 2025
2fffa6f
simplify
glyh May 27, 2025
1d74dab
a
glyh May 27, 2025
9b40b69
update
glyh May 27, 2025
5db7123
update
glyh May 27, 2025
6d30771
upd
glyh May 30, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
snark_work_lib/dune: run dune fmt --auto-promote
  • Loading branch information
dannywillems authored and glyh committed Apr 29, 2025
commit edadc19e8212f0dedcea4057ffb69913dedd8cad
1 change: 0 additions & 1 deletion src/lib/snark_work_lib/dune
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@
signature_lib
transaction_snark
ppx_version.runtime)

(preprocess
(pps ppx_jane ppx_deriving_yojson ppx_version))
(instrumentation
Expand Down