Support deletion of complete rows

[mmois]

Is there a strong reason for not being able to delete a complete row?

Currently the method HaeinsaTable.delete() checks that the size of the family map is not zero and throws an exception if that is the case. The comment above the precondition check states that it is not possible to delete entire rows because of the lock column.

But shouldn't it be possible to remove the complete row in the HaeinsaTable.makeStable() method? Instead of setting the lock state for this row to STABLE, the row is removed from HBase.

In case the row is a secondary row in the transaction, the primary row needs the information that this row needs to be deleted. During transaction recovery (in the state COMMITTED) it can check whether the row has been deleted or not. It can delete the row if the prewriteTimestamp matches (in order to not delete a concurrently added new version).

In case the row is the primary row in the transaction, it can be removed completely.

If there are two competing transactions, one trying to delete and the other trying to update the row, then the updating one would fail (if the other one was faster) because the checkAndPut() operation fails when the row no longer exists. For the other way round HBase's checkAndDelete() operation can be used.

[eincs]

Thank you for suggestion. At a glance, it seems to be possible to support deletion of whole row in Haeinsa with your solution. I will look deeper into the solution over this weekend.

[eincs]

Sorry for replying late.

I carefully investigated on this issue, and conclusion is: It is not able to support deletion of row with your solution. It is related with behavior of deletion of HBase: On executing delete operation, Hbase does not delete data immediately, it just leave delete tombstone. And it actually delete data during compaction.

Let assume that there several client on different machine are accessing to Hbase via Haeinsa. These local time of these machine can be different with other machines. In Haeinsa, to each client uses own local time of the machine to determine commit timestamp of the transaction. To adjust difference of local times of each machine, Haeinsa uses max value of all timestamps on each lock columns of the transaction.
See: HaeinsaTransaction.java#L232

So, delete entire row and not leaving lock column will cause problem: Deleted value can be read by client with younger local time, when accessing to the row after deleted.

[eincs]

There can be several possible solution to support deleting whole row in Haeinsa:

• On deletion of row, just mark row as deleted on lock column, and delete actual data during major compaction. HBase supports running user defined codes on major compaction with coprocessors.
• Do not rely only on timestamp on lock column to adjust timestamp of the transaction: use timestamp from something like timestamp oracle.

I will investigate further on this issue in the future.

[saadmufti]

Is it possible to do this if the transaction is just a single delete with the size of the family map being 0 (indicating desire to delete the entire row) and there is only a single row involved in the transaction? Then it would seem to be safe to a) do a read the current state of the '!lock!' column, if any, b) do one checkAndDelete that deletes all column families, including the '!lock!' column. I don't see why the concern about differing client clocks applies. The checkAndDelete will only work if the current value of !lock! matches what was read based on the server time at the time of the checkAndDelete. Even if a concurrent transaction from another client is trying to write new data, it would do a checkAndPut which would only succeed if it matches the value of the lock column read in that transaction, again atomically based on server time.

[saadmufti]

The intent of all this is I'm hoping it would be easy to optimize for a delete of the whole row if it is a single row with no secondary row.

[saadmufti]

Similar to what is already done to optimize for the only mutation being a put on a single row.

[saadmufti]

In case anyone is interested, we have implemented this in our fork https://github.com/aol/haeinsa in the aol_master branch. We're using it in production and so far haven't seen any issues. It improved our performance when deleting single rows by a lot.

Cheers.