DB3: alexander-akhmetov/raft-kv
While the RheaKV work is unfinished, I was exploring study resources from other contributors. I got introduced to TinyKV & TinySQL offered by the PingCAP team. This was written in Go and apparently, Go had more market value than Rust. Learning Go was getting prioritized on my to-do list. I was still looking for a smaller code base to start with and see, how Go databases work. That's when RaftKV hit my path. I am about to dissect the codebase of RaftKV to help me understand it better.
Note: I have made some minor changes (updating logger to hclog & using go.mod) in my fork of Raft KV.
Main.go
This is the driver class.
1) Command Line Arguments
2.a) go-flags
This library provides similar functionality to the built-in flag library of go but provides much more functionality and more excellent formatting.
Package flags provide an extensive command-line option parser. The flags package is similar in functionality to the go builtin flag package but offers more options and uses reflection to provide a convenient and succinct way of specifying command-line options
2.b) go-flags syntax
3. a) Hashicorp RAFT
The Go based and C based backend implementation for the raft
We are using raft-boltdb (for Go)
We could also use, raft-mdb (for C)
3.b) Protocol
3.c) raft-grpc-example
4.a) Imports
Let's start with the imports. We see, that we use
fmt: for printing
log: for logging
os: for getting args
time: for using time.Seconds
github.com/jessevdk/go-flags : for parsing CLI args
4.b) Reading CLI args
The Options are specified as ENV properties in the docker-compose.yaml
Node 1Config containsBootStrapset totrue.- It doesn’t contain
Joinenv parameter.
Quoting the comments
- Also, the leader node must start a web interface so other nodes will be able to join the cluster.
- They send a POST request to the leader with their IP address, and it adds them to the cluster (Shown in
Section: Cluster Join)
Something like this
We made Node 1 as a leader.
Node 2 & 3 are similar
They are dependent on Node 1. They have Join parameter.
4.c) Create a new Node
4.d) Passing Storage to a go-routine for a continuous status check
It would print something like this
4.e) Cluster Join
4.f) Start Gin Server
node/node.go
This is the class that will interact with the raft-boltdb.
1.a) Imports
bytes: convert text to bytes so that it can passed via POST callencoding/json: returns json encoding of objectnet: for resolving TCP Addressnet/http: For making a POST callpath/filepath: Used to join two path namesgithub.com/hashicorp/raft: RAFT protocolgithub.com/hashicorp/raft-boltdb: RAFT backend in Go- github.com/hashicorp/go-hclog: Used for
Here we are assigning a name (rbolt) for the raft-boltdb import.
1.b) Config struct
2.a) NewRStorage — Create RStorage and MakeDir
In the main.go, we are using the below code.
storage, err := node.NewRStorage(&config)
2.b) Create a new RAFT node
Here, you can see that
rboldis used to create a new Store having paths: “temp/raft-log.bolt”, “temp/raft-stable.bold”filepathis used toJoinDirectory paths.hclog: We passhc_loggerand pass it toraftConfig.Logger = loggerraftTransport(): This function is invoked to create a TCP transport layer
2.c) Setting Bootstrap Config in RaftNode, and Setting that in RStorage
Ultimately we are creating RStorage. We will set the RStorage raftNode to be the updated RaftNode containing the Bootstrap configuration (ie Servers[]).
2.d) raftTransport()
Here we are creating a NewTCPTransportWithLogger
2.e)JoinCluster
Deferin GoLang
Defer body.closeafter receiving response
2.f) GetClusterServers()
- Reading configs from
RaftNode - Used in
http.go
2.g) AddVoter
/node/storage.go
Storage.go imports “raft” package and thereby implements the FSM interface for RStorage and FSMSnapshot interface for fsmSnapshot.
1.a) Imports
1.b) RStorage struct
1.c) Get
As you can see, we are reading from the fieldstorage map directly. ie from memtable (save in the application state)
1.d) Set
- Here we have
logEventstruct which is used to hold, thecommand - Inside Set(), we are using
sync.Mutexvariable, to lock. - We
deferunlock, until the current task is completed. - Now we marshal the object to JSON.
- and finally, we do,
RaftNode.Apply(data, timeout) - The
RaftNode, as seen thenode.gocontainsraft-boltdb, which is used in thelogStore&stableStore.
2.a) Interface
Now, we will sidetrack a bit and learn about Interface in Go.
3.a) FSM Interface
Apply()
Note, RaftNode.Apply() is not the same as this FSM Apply()
Snapshot()
Restore()
These functions are invoked at fsm.go inside the raft package.
4.a) FSM Implementation in storage.go
- Apply()
It first locks using the mutex, then writes the value to the in-memory map
- Snapshot()
Creates a copy of the in-memory map and returns it in the form of fsmSnapshot
- Restore()
5.a) FSM Snapshot
This is the core element, which does the compaction.
6.a) fsmSnapshot in storage.go
Note that, we are using the first letter to a small case, to indicate that this will be used only within the storage.go
We are now implementing the FSMSnapshot interface.
- Persist()
I think the sink here is backed by boltdb mentioned in the node.go
- Release()
server/http.go
The main Gin server.
1.a) Imports
1.b) Setting router
1.c) Get Key View
Here we are reading the value from storage. We are using gin.Context to read the parameter, and return back a JSON response.
1.d) setKeyView
Below is the struct for setKeyData.
- We are using ` for specifying metadata. in our case, the JSON field name.(TAG)
- Back Quote in String
Below is the code for the setKeyView function
- Here we are using the
structsetKeyData. - we are reading
parameterfromgin.Context - we use
BindJSONto read the POST request JSON into the setKeyData object - We use
storage.set()to set the value.
1.e) JoinView
Here you can see that
storage.AddVoter()is taken fromnode.go, whereasstorage.Get()is taken fromstorage.go
So, how is it working?
Lets now, learn about Methods.
2.a) Methods vs Functions
3.a) Simple Example
server/http_test.go
4.a) Imports
os: used for removing files in the directory
net/http /httptest: httptest.NewRecorder() used for recording response
http: used for creating request
testing: used for testing
github.com/stretchr/testify/assert: used for assertion
4.b) Functions
- init()
- getLeaderNode()
- setupNode()
- TestGetValue()
- PerformRequest and assertValue
Conclusion:
This completes a raft-kv, a simple raft based database written in Go. The Logs are saved in BoldDB and uses RAFT for replication. This was a great experience and counts towards my journey in learning Database Internal.
