Skip to content

Lambdas are not serialisable #4442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
allanrenucci opened this issue May 2, 2018 · 1 comment
Closed

Lambdas are not serialisable #4442

allanrenucci opened this issue May 2, 2018 · 1 comment
Assignees
@smarter
Labels
area:transform itype:bug

Comments

@allanrenucci
Copy link
Contributor

allanrenucci commented May 2, 2018 •
edited
Loading 

object Test {
  def serializeDeserialize[T <: AnyRef](obj: T): T = {
    import java.io._
    val buffer = new ByteArrayOutputStream
    val out = new ObjectOutputStream(buffer)
    out.writeObject(obj)
    val in = new ObjectInputStream(new ByteArrayInputStream(buffer.toByteArray))
    in.readObject.asInstanceOf[T]
  }

  def main(args: Array[String]): Unit = {
    val adder = serializeDeserialize((x: Int) => x + 1)
    assert(adder(1) == 2)
  }
}

> dotc -d out Test.scala
> dotr -classpath out Test
Exception in thread "main" java.io.NotSerializableException: Test$$$Lambda$1/1023892928
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
	at Test$.serializeDeserialize(Test.scala:6)
	at Test$.main(Test.scala:12)
	at Test.main(Test.scala
@smarter
Copy link
Member

smarter commented May 2, 2018

This is a big piece of work, check for deserializeLambda in scalac to know more: https://github.com/scala/scala/search?utf8=%E2%9C%93&q=deserializeLambda&type=

@smarter smarter self-assigned this Jan 26, 2019
smarter added a commit to smarter/dotty that referenced this issue Feb 2, 2019
@smarter
Fix scala#4442: Make lambdas serializable
be3c312
smarter added a commit to smarter/dotty that referenced this issue Feb 2, 2019
@smarter
Fix scala#4442: Make lambdas serializable
649bdc5
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Make lambdas serializable
2bf2215 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
78cb37d 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
c4c930f 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
e9971c4 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
ff4c5cf 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
baa05cd 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
4c863d7 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
73cf5f0 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
e50de50 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 3, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
beaed8c 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
smarter added a commit to dotty-staging/dotty that referenced this issue Feb 6, 2019
@smarter @retronym @lrytz
Fix scala#4442: Add support for lambda serialization
97ffc24 
In Scala, lambdas whose SAM extend Serializable as well as lambdas whose
SAM is simply scala.Function* should be serializable but this was not
the case in Dotty so far. On the JVM, lambdas instantiated using
invokedynamic calls require some special handling to be serializable:

1. We need to use the invokedynamic bootstrap method
   `LambdaMetaFactory#altMetafactory` instead of
   `LambdaMetaFactory#metafactory`, this allows us to pass the
   FLAG_SERIALIZABLE flag. This is implemented in the backend submodule
   commit included in this commit (see lampepfl/scala#39).

2. In the enclosing class where the lambda is defined, a
   $deserializeLambda$ method needs to be generated, this is implemented
   in this commit.

Most of the logic for $deserializeLambda$ is implemented in the Scala
2.12 standard libraries class scala.runtime.LambdaDeserialize and
scala.runtime.LambdaDeserializer which can be used here as-is, the only
logic we actually need to implement here is:

1. In `collectSerializableLambdas`, we collect all serializable lambdas.
   Unlike scalac, our backend does not do any inlining currently so our
   implementation is more straightfoward than theirs.

2. In `addLambdaDeserialize`, we implement the actual
   $deserializeLambda$ method, the implementation here is directly
   copied from scalac, it's complex because it needs to work around a
   limitation of bootstrap methods (they cannot take more than 251
   arguments).

Since some of this code comes from scalac, this is:
Co-Authored-By: Jason Zaugg <[email protected]>
Co-Authored-By: Lukas Rytz <[email protected]>
@smarter smarter closed this as completed in 5fc2119 Feb 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@smarter smarter

Labels
area:transform itype:bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@smarter @allanrenucci