[WIP] Optimization: Improve performance for repetitive lookup

CacheLookupTable leaves the bucket vacant when the object is freed.
Because of this, in the worst case, subscript have to perform full linear
search for all buckets.

Let's say we have a table like this:

  bucket 0: obj1(hash: 0)
  bucket 1: obj2(hash: 1)
  bucket 2: nil
  bucket 3: obj3(hash: 0)
  bucket 4: nil

If obj1 is freed, it become:

  bucket 0: nil
  bucket 1: obj2(hash: 1)
  bucket 2: nil
  bucket 3: obj3(hash: 0)
  bucket 4: nil

When looking-up obj3, the search starts from "bucket 0", thus it needs 3
iteration.
This patch moves the obj3 to "bucket 0" the first nil hole found.
So the next lookup doesn't need any iteration.

  bucket 0: obj3(hash: 0)
  bucket 1: obj2(hash: 1)
  bucket 2: nil
  bucket 3: nil
  bucket 4: nil

Author: rintaro

Sources/SwiftSyntax/CacheLookupTable.swift

@@ -197,15 +197,31 @@ class CacheLookupTable<T: Identifiable & AnyObject> {
   /// insert()-ed or it's already been freed.
   public subscript(id: T.Identifier) -> T? {
     // Since we don't fill the bucket when the value is freed (because we don't
-    // know), we can't stop iteration a hole. So in the worst case (i.e. if the
-    // object is not contained in this table), full linear search is required.
+    // know), we can't stop iteration at a hole. So in the worst case (i.e. if
+    // the object doesn't exist in the table), full linear search is needed.
     // However, since we assume the value exists and hasn't been freed yet,
     // we expect it's stored near the 'idealBucket' anyway.
     let idealBucket = _idealBucket(for: id)
     var bucket = idealBucket
+    var firstHole: Int? = nil
     repeat {
-      if buckets[bucket].value?.id == id {
-        return buckets[bucket].value
+      if let obj = buckets[bucket].value {
+        if obj.id == id {
+          if let firstHole = firstHole {
+            // Move the object to the first hole found. This improves lookup
+            // performance in the next lookup.
+            let holeP = buckets + firstHole
+            let bucketP = buckets + bucket
+
+            let tmp = holeP.move()
+            holeP.moveInitialize(from: bucketP, count: 1)
+            bucketP.initialize(to: tmp)
+          }
+          return obj
+        }
+      } else if firstHole == nil {
+        // Remember the first hole found.
+        firstHole = bucket
       }
       bucket = (bucket &+ 1) & _bucketMask
     } while bucket != idealBucket