Commit graph

13 commits

Author SHA1 Message Date
Luke Parker
32435d8a4c
Consolidate RockDB code
Moves explicitly to zstd. RockDB recommends zstd, or at least lz4 over snappy,
and this minimizes which dependencies we pull in.
2023-07-25 21:43:27 -04:00
Luke Parker
624fb2781d
Update how RPCs are handled
The processor now takes three vars and joins them itself. message-queue uses a
single argument, with defaults, as it's a service we control.
2023-07-21 14:01:42 -04:00
Luke Parker
9effd5ccdc
Add a Docker-based test for the message-queue service 2023-07-20 18:53:11 -04:00
Luke Parker
acc9495429
Use MessageQueue instead of MemCoordinator in processor
Also has it use RocksDB.
2023-07-17 18:02:29 -04:00
Luke Parker
344ac9cbfc
Add ack signatures
Also modifies message signatures to be binding to from, not just from's key.
2023-07-17 17:40:34 -04:00
Luke Parker
6ccac2d0ab
Add a message-queue connection to processor
Still needs love, yet should get us closer to starting testing.
2023-07-17 15:49:17 -04:00
Luke Parker
712f11d879
Correct the message-queue's handling of var 2023-07-17 02:01:31 -04:00
Luke Parker
0a367bfbda
Add common crate to access env variables
In the future, we should use a proper secret store (not just env variables).
This lets us update one block of code and not n in the future.
2023-07-17 00:53:05 -04:00
Luke Parker
845c2842b5
message-queue RocksDB + fix listening 2023-07-17 00:22:55 -04:00
Luke Parker
8543487db2
Document message-queue RPC methods 2023-07-16 20:53:58 -04:00
Luke Parker
a9072e6b1b
Remove signature from get_next_message
Duee to signature replaying, it's very annoying to provide meanigful data
access privacy. None of these messages should be private/have sensitive data
anyways though.
2023-07-16 20:38:13 -04:00
Luke Parker
b0c28a1cf0
Move message_queue over to deduplication via intents
Due to each service having multiple distinct clocks, we can't expect a stable
ordering except the ordering an intact message-queue provides. The messages
emitted should be consistent however, solely with unknown order, which is why
we can craft intents based on their contents (already implemented by
processor-messages).
2023-07-16 20:34:35 -04:00
Luke Parker
6267acf3df
Add a message queue
This is intended to be a reliable transport between the processors and
coordinator. Since it'll be intranet only, it's written as never fail.

Primarily needs testing and a proper ID.
2023-07-01 08:53:46 -04:00