Eventual consistency via IMRS clustering proposal

[Xe]

In the beginning there was chaos. Database servers were overbloated, overcomplicated, and fully did a great job of abstracting away the /actual data/ that developers want to see into tables, bureaucracy, and pain.

There was a hero. Its name is OlegDB. OlegDB for a while touted its lack of multi-master replication or other complicated shit like that.

No more.

I propose that OlegDB be made eventually consistent across a cluster by applying the principles of MAYO on top of IMRS (INTERCAL/Malbolge Reliable Systems) to allow for an infinite level of scalability that our database needs to be scalable to all kinds of production or staging needs.

.

The API

So basically servers would be able to at runtime form a group of eventually consistent IMRS nodes. When a server gets a request to read a key that it doesn't have, it will ask its peers if anyone else has it. If it does it will use that as the canonical value for the key.

If there is a collision, this is where the real fun begins. The HoL of the IMRS cluster will call an immediate halt to all reads and writes to and from the database so that the deathmatch between opposing keys and values can begin.

Deathmatch Process

The two opposing servers will contact eachother with the value of the key they think is right and then also include a timestamp formatted in terms of the number of seconds since time index zero, midnight at Feburary 6, 1966. The servers will then send eachother the values to a /_imrs/deathmatch route, where both servers will then calculate the applicative sum of both values and timestamps. They will then translate that to trinary and do the crazy operation against both sets of timestamps and values.

If the result is 0, the challenger wins

If the result is 1, the defender wins

If the result is 2, the winner is chosen by a coin toss

The "winning" key->value pair gets put into the database and is broadcast to the other nodes in the group as the correct value.

---

Thoughts?

ping @kyleterry @qpfiffer @dequis

[Hamcha]

I was more for a "Capture the Flag" kind of approach to contention, but this can work too I guess.

[lykkin]

i'd like to see a proof of correctness before 📦

[qpfiffer]

Fits Oleg pretty damn well. I think this is probably blocked by the memory leaks in the frontend, and the better transactions stuff.

[sariyamelody]

I'd like to propose a challenger.

RACP

Really Awesome Clustering Protocol, Yeah!

The general idea is that we get an MS SQL database, and then on every node in a cluster, we generate arbitrary code and insert it into a single table. To make a decision on which code to execute on the cluster, we run SELECT code FROM table ORDER BY date LIMIT 1 upon the database. This is the sole conflict resolution outcome, and I think it's fundamentally more webscale and agile than this solution.

Thoughts?

[qpfiffer]

Where the hell are we going to get an MS SQL license you fascist? This is codesmell. I will not stand for this intolerable propriety.

Edit: And your acronym doesn't even include 'Yeah!'

[Xe]

RACPY is better, @skyhighwings, would RACPY work better?

[sariyamelody]

@Xe Yes.
@qpfiffer We can just use the trial! I'm sure Microsoft won't mind.

[Xe]

📦