Moving to new Salsa version using query-group repo (DRAFT)

This is a WIP
* Using an updated version of query-group to better support lifetimes.
* Embedded query-group in cairo-lang-proc-macros.
* cairo-lang-proc-macro's derive debug with db now supprots generics better.
* Using salsa::Update on non salsa structs to get the db to work correctly.
* Moved some SmolStr and Arc<str> to use be interned for salsa::Update support.
* Implemented Ord for SmolStrId to have BTreeMap's salsa::Update work.
* Removing upcasting and 'static where applicable
* Adding alot of <'_> for a happy clippy.
* Debug with db now supports lifetimes. Fixed implementations across the codebase.

* Upgraded cairo-lang-debug by:
    1. saying Db has the salsa::Database trait
    2. making Dummy in test not use shortid
    3. Debug with Db can make some trouble but using .long where needed fixes it.
* cairo-lang-utils:
    1 .The short_id macro is not really needed so I gutted it and have short_id double interned
    1.1. I made short_id have a single long field that is returned by ref.
    2. Implemented salsa::Update for OrderedHashMap.
    3. Implemented salsa::Update for OrderedHashSet.
    4. Added an as_intern_id and from_intern_id using asid and fromid
    5. No need for upcast to be static
* cairo-lang-filesystem
    1. Removed all interned valued from input.
    2. Updated code with lifetimes to better support <'db> interned values.
    3. Tests needed updating as set methods are a bit more tricky now with lifetimes.
    4. Added utility function for setting input values with using lifetimed objects.
    5. Added macro_rules! to set_crate_config and override_file_content so they still accept a short id.
* cairo-lang-syntax:
    + Updated all code to use lifetimes.
    + Updated all code to use the new query-group.
    + Updated codegen.
    + Updated ast.
* cairo-lang-diagnostics:
    + Updated all code to use lifetimes.
    + Updated all code to use the new query-group.
    + Added lifetime to DiagnosticEntry and DiagnosticBuilder as it was necessary by plugin tests.
* cairo-lang-parser:
    + Updated all code to use lifetimes.
    + Updated all code to use the new query-group.
    + Removed the references to empty data structs as they are no longer needed.
    + Using long to turn StrId into a ref for the parser.
* cairo-lang-sierra
    + No need for internkey (as query group will use #[salsa::interned] instead)
* cairo-lang-defs
    + Added lifetime to db
    + Added lifetime to diagnostic_utils
    + Added lifetime to ids
    + Fixing macros in ids to support lifetimes
    + Using self.long(db) instead of lookup_intern to get a ref with a 'db lifetime
    + Change from Arc<[T]> to Arc<Vec<T>> which allows for salsa::Update.
    + Tests to use macro_rules! to get lifetimes right.
    + Updated tests to work with all the changes
    + Fixed debug print formatting.
* cairo-lang-plugins:
    + Updated all code to use lifetimes.

Added preparation for semantic db.
Debug lifetime fixes.
Cycle support for queries
More salsa::Update support.
More lifetimes.
Update for unordered_hash_map and unordered_hash_set.
Deque wrapper and Update for it.
Some rewriter macro support.
Moved priv queries back to salsa db - using update on FunctionBodyData
    - Update assumes that ExprId and PatternId are from the FunctionBody in the struct.

AI helped a bit:
Core Upgrade
Migrated to a new Salsa version with better lifetime support by embedding an updated query-group implementation directly into cairo-lang-proc-macros
Added comprehensive lifetime annotations (<'db>, <'_>) throughout the codebase to support the new Salsa architecture
Key Technical Changes
Salsa Integration Improvements:
Implemented salsa::Update trait for non-Salsa structs (OrderedHashMap, OrderedHashSet, Deque wrapper)
Converted SmolStr and Arc<str> to interned values for Salsa compatibility
Added Ord implementation for SmolStrId to support BTreeMap's salsa::Update
Database & Query System:
Removed static lifetime requirements and upcasting where possible
Added cycle support for queries
Improved generic support in proc-macro's derive debug with db
Major Crate Updates:
cairo-lang-proc-macros: Added 11 new test files and embedded query-group functionality
cairo-lang-utils: Gutted short_id macro, added Deque wrapper, intern conversion utilities
cairo-lang-filesystem: Removed interned values from input, added utility functions for lifetime-aware inputs
All syntax/parser/semantic crates: Updated to use explicit lifetimes throughout
New Functionality
Added preparation for semantic database
Improved macro support for lifetimes in test utilities
Changed from Arc<[T]> to Arc<Vec<T>> for salsa::Update compatibility
This is a foundational change that modernizes the entire Cairo compiler's incremental computation infrastructure to better handle borrowed data and lifetimes.

commit-id:9e4ac020

Author: eytan-starkware

Cargo.lock

@@ -18,7 +18,7 @@ opt-level = 3
 [profile.ci-dev.package."bimap"]
 
 opt-level = 3
-[profile.ci-dev.package."rust-analyzer-salsa"]
+[profile.ci-dev.package."salsa"]
 opt-level = 3
 
 [workspace]
@@ -101,7 +101,9 @@ derivative = "2.2.0"
 diffy = "0.4.2"
 env_logger = "0.11.6"
 genco = "0.17.10"
-good_lp = { version = "1.11.0", features = ["minilp"], default-features = false }
+good_lp = { version = "1.11.0", features = [
+    "minilp",
+], default-features = false }
 hashbrown = "0.15.2"
 id-arena = "2.2.1"
 ignore = "0.4.23"
@@ -125,10 +127,13 @@ quote = "1.0.38"
 rand = "0.9.0"
 rayon = "1.10.0"
 rstest = "0.25.0"
-salsa = { package = "rust-analyzer-salsa", version = "0.17.0-pre.6" }
+salsa = "0.22.0"
 schemars = { version = "0.8.21", features = ["preserve_order"] }
 semver = { version = "1.0.25", features = ["serde"] }
-serde = { version = "1.0.217", default-features = false, features = ["derive","rc"] }
+serde = { version = "1.0.217", default-features = false, features = [
+    "derive",
+    "rc",
+] }
 serde_json = "1.0.138"
 typetag = "0.2"
 sha2 = "0.10.8"

Cargo.toml

@@ -8,9 +8,9 @@ description = "Debug utilities for query objects."
 
 [dependencies]
 cairo-lang-utils = { path = "../cairo-lang-utils", version = "~2.11.4" }
+salsa.workspace = true
 
 [dev-dependencies]
-cairo-lang-proc-macros = { path = "../cairo-lang-proc-macros"}
+cairo-lang-proc-macros = { path = "../cairo-lang-proc-macros" }
 env_logger.workspace = true
-salsa.workspace = true
 test-log.workspace = true

crates/cairo-lang-debug/Cargo.toml

@@ -11,28 +11,28 @@ use std::sync::Arc;
 use cairo_lang_utils::ordered_hash_map::OrderedHashMap;
 use cairo_lang_utils::ordered_hash_set::OrderedHashSet;
 
-pub trait DebugWithDb<Db: ?Sized> {
-    fn debug<'me, 'db>(&'me self, db: &'me Db) -> DebugWith<'me, Db>
+pub trait DebugWithDb<'db, Db: ?Sized> {
+    fn debug<'me>(&'me self, db: &'db Db) -> DebugWith<'me, 'db, Db>
     where
         Self: Sized + 'me,
     {
         DebugWith { value: BoxRef::Ref(self), db }
     }
 
     #[allow(dead_code)]
-    fn into_debug<'me, 'db>(self, db: &'me Db) -> DebugWith<'me, Db>
+    fn into_debug<'me>(self, db: &'db Db) -> DebugWith<'me, 'db, Db>
     where
         Self: Sized + 'me,
     {
         DebugWith { value: BoxRef::Box(Box::new(self)), db }
     }
 
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result;
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result;
 }
 
-pub struct DebugWith<'me, Db: ?Sized> {
-    value: BoxRef<'me, dyn DebugWithDb<Db> + 'me>,
-    db: &'me Db,
+pub struct DebugWith<'me, 'db, Db: ?Sized> {
+    value: BoxRef<'me, dyn DebugWithDb<'db, Db> + 'me>,
+    db: &'db Db,
 }
 
 enum BoxRef<'me, T: ?Sized> {
@@ -51,137 +51,137 @@ impl<T: ?Sized> std::ops::Deref for BoxRef<'_, T> {
     }
 }
 
-impl<D: ?Sized> std::fmt::Debug for DebugWith<'_, D> {
+impl<D: ?Sized> std::fmt::Debug for DebugWith<'_, '_, D> {
     fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
         DebugWithDb::fmt(&*self.value, f, self.db)
     }
 }
 
-impl<Db: ?Sized, T: ?Sized> DebugWithDb<Db> for &T
+impl<'db, Db: ?Sized, T: ?Sized> DebugWithDb<'db, Db> for &T
 where
-    T: DebugWithDb<Db>,
+    T: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         T::fmt(self, f, db)
     }
 }
 
-impl<Db: ?Sized, T: ?Sized> DebugWithDb<Db> for Box<T>
+impl<'db, Db: ?Sized, T: ?Sized> DebugWithDb<'db, Db> for Box<T>
 where
-    T: DebugWithDb<Db>,
+    T: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         T::fmt(self, f, db)
     }
 }
 
-impl<Db: ?Sized, T> DebugWithDb<Db> for Rc<T>
+impl<'db, Db: ?Sized, T> DebugWithDb<'db, Db> for Rc<T>
 where
-    T: DebugWithDb<Db>,
+    T: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         T::fmt(self, f, db)
     }
 }
 
-impl<Db: ?Sized, T: ?Sized> DebugWithDb<Db> for Arc<T>
+impl<'db, Db: ?Sized, T: ?Sized> DebugWithDb<'db, Db> for Arc<T>
 where
-    T: DebugWithDb<Db>,
+    T: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         T::fmt(self, f, db)
     }
 }
 
-impl<Db: ?Sized, T> DebugWithDb<Db> for [T]
+impl<'db, Db: ?Sized, T> DebugWithDb<'db, Db> for [T]
 where
-    T: DebugWithDb<Db>,
+    T: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         let elements = self.iter().map(|e| e.debug(db));
         f.debug_list().entries(elements).finish()
     }
 }
 
-impl<Db: ?Sized, T> DebugWithDb<Db> for Vec<T>
+impl<'db, Db: ?Sized, T> DebugWithDb<'db, Db> for Vec<T>
 where
-    T: DebugWithDb<Db>,
+    T: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         let elements = self.iter().map(|e| e.debug(db));
         f.debug_list().entries(elements).finish()
     }
 }
 
-impl<Db: ?Sized, T> DebugWithDb<Db> for Option<T>
+impl<'db, Db: ?Sized, T> DebugWithDb<'db, Db> for Option<T>
 where
-    T: DebugWithDb<Db>,
+    T: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         let me = self.as_ref().map(|v| v.debug(db));
         std::fmt::Debug::fmt(&me, f)
     }
 }
 
-impl<Db: ?Sized, K, V, S> DebugWithDb<Db> for HashMap<K, V, S>
+impl<'db, Db: ?Sized, K, V, S> DebugWithDb<'db, Db> for HashMap<K, V, S>
 where
-    K: DebugWithDb<Db>,
-    V: DebugWithDb<Db>,
+    K: DebugWithDb<'db, Db>,
+    V: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         let elements = self.iter().map(|(k, v)| (k.debug(db), v.debug(db)));
         f.debug_map().entries(elements).finish()
     }
 }
 
-impl<Db: ?Sized, K, V> DebugWithDb<Db> for BTreeMap<K, V>
+impl<'db, Db: ?Sized, K, V> DebugWithDb<'db, Db> for BTreeMap<K, V>
 where
-    K: DebugWithDb<Db>,
-    V: DebugWithDb<Db>,
+    K: DebugWithDb<'db, Db>,
+    V: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         let elements = self.iter().map(|(k, v)| (k.debug(db), v.debug(db)));
         f.debug_map().entries(elements).finish()
     }
 }
 
-impl<Db: ?Sized, K: Hash + Eq, V> DebugWithDb<Db> for OrderedHashMap<K, V>
+impl<'db, Db: ?Sized, K: Hash + Eq, V> DebugWithDb<'db, Db> for OrderedHashMap<K, V>
 where
-    K: DebugWithDb<Db>,
-    V: DebugWithDb<Db>,
+    K: DebugWithDb<'db, Db>,
+    V: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         let elements = self.iter().map(|(k, v)| (k.debug(db), v.debug(db)));
         f.debug_map().entries(elements).finish()
     }
 }
 
-impl<Db: ?Sized, A> DebugWithDb<Db> for (A,)
+impl<'db, Db: ?Sized, A> DebugWithDb<'db, Db> for (A,)
 where
-    A: DebugWithDb<Db>,
+    A: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         f.debug_tuple("").field(&self.0.debug(db)).finish()
     }
 }
 
-impl<Db: ?Sized, A, B> DebugWithDb<Db> for (A, B)
+impl<'db, Db: ?Sized, A, B> DebugWithDb<'db, Db> for (A, B)
 where
-    A: DebugWithDb<Db>,
-    B: DebugWithDb<Db>,
+    A: DebugWithDb<'db, Db>,
+    B: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         f.debug_tuple("").field(&self.0.debug(db)).field(&self.1.debug(db)).finish()
     }
 }
 
-impl<Db: ?Sized, A, B, C> DebugWithDb<Db> for (A, B, C)
+impl<'db, Db: ?Sized, A, B, C> DebugWithDb<'db, Db> for (A, B, C)
 where
-    A: DebugWithDb<Db>,
-    B: DebugWithDb<Db>,
-    C: DebugWithDb<Db>,
+    A: DebugWithDb<'db, Db>,
+    B: DebugWithDb<'db, Db>,
+    C: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         f.debug_tuple("")
             .field(&self.0.debug(db))
             .field(&self.1.debug(db))
@@ -190,31 +190,31 @@ where
     }
 }
 
-impl<Db: ?Sized, V, S> DebugWithDb<Db> for HashSet<V, S>
+impl<'db, Db: ?Sized, V, S> DebugWithDb<'db, Db> for HashSet<V, S>
 where
-    V: DebugWithDb<Db>,
+    V: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         let elements = self.iter().map(|e| e.debug(db));
         f.debug_list().entries(elements).finish()
     }
 }
 
-impl<Db: ?Sized, V> DebugWithDb<Db> for BTreeSet<V>
+impl<'db, Db: ?Sized, V> DebugWithDb<'db, Db> for BTreeSet<V>
 where
-    V: DebugWithDb<Db>,
+    V: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         let elements = self.iter().map(|e| e.debug(db));
         f.debug_list().entries(elements).finish()
     }
 }
 
-impl<Db: ?Sized, V: Hash + Eq> DebugWithDb<Db> for OrderedHashSet<V>
+impl<'db, Db: ?Sized, V: Hash + Eq> DebugWithDb<'db, Db> for OrderedHashSet<V>
 where
-    V: DebugWithDb<Db>,
+    V: DebugWithDb<'db, Db>,
 {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &Db) -> std::fmt::Result {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>, db: &'db Db) -> std::fmt::Result {
         let elements = self.iter().map(|e| e.debug(db));
         f.debug_list().entries(elements).finish()
     }
@@ -238,9 +238,9 @@ pub mod helper {
 
     pub struct HelperDebug<T, Db: ?Sized>(PhantomData<T>, PhantomData<Db>);
 
-    impl<T: DebugWithDb<Db>, Db: ?Sized> HelperDebug<T, Db> {
+    impl<'db, T: DebugWithDb<'db, Db>, Db: ?Sized> HelperDebug<T, Db> {
         #[allow(dead_code)]
-        pub fn helper_debug<'a, 'b: 'a>(a: &'a T, db: &'b Db) -> DebugWith<'a, Db> {
+        pub fn helper_debug<'a>(a: &'a T, db: &'db Db) -> DebugWith<'a, 'db, Db> {
             a.debug(db)
         }
     }