Documentation generation broken by merging #3239 with typer error

[smarter]

Somehow, merging #3239 broke the documentation generation with the following error:

-- Error: /drone/src/github.com/lampepfl/dotty/doc-tool/src/dotty/tools/dottydoc/model/comment/HtmlParsers.scala:96:103 
96 |          s"<dl>${items map { case (t, d) => s"<dt>${inlineToHtml(t)}</dt><dd>${blockToHtml(d)}</dd>" } }</dl>"
   |                                                                                                       ^
   |ambiguous implicits: both method OnceCanBuildFrom in object TraversableOnce and method canBuildFrom in object GenIterable match type scala.collection.generic.CanBuildFrom[
   |  scala.collection.SortedMap[dotty.tools.dottydoc.model.comment.Inline, 
   |    dotty.tools.dottydoc.model.comment.Block
   |  ]
|, String, That] of parameter bf of method map in trait TraversableLike

(see http://dotty-ci.epfl.ch/lampepfl/dotty/627/8), you can reproduce the failure by running:

sbt genDocs

[smarter]

/cc @odersky

[smarter]

This can be reproduced by just running dotc with the right set of files: https://gist.github.com/78336fe64cab378df70d300a92c97d94 However this is incredibly hard to minimize: the reproduction need to jointly compile more than 300 files, remove any random subset of files from this list makes the error go away.

[odersky]

I can't reproduce this on latest master anymore.

[smarter]

It's still broken on the CI: http://dotty-ci.epfl.ch/lampepfl/dotty/728/8 and on my side I can still reproduce it with both sbt genDocs and the script above.

[smarter]

Here's something interesting I found by staring at output logs, when we get the ambiguous implicit error, the companions for Iterable are wrong for some reason:

[log frontend]         ==> collectCompanions(scala.collection.Iterable[(scala.collection.Map[Map.this.K, Map.this.V](Map.this), scala.collection.Map[Map.this.K, Map.this.V](Map.this))])?

[log frontend]           ==> collectCompanions(Tuple2)?

[log frontend]             ==> collectCompanions(Product2[(Tuple2.this.T1, Tuple2.this.T2)(Tuple2.this), (Tuple2.this.T1, Tuple2.this.T2)(Tuple2.this)])?

[log frontend]               ==> collectCompanions(Product2)?

[log frontend]               <== collectCompanions(Product2) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(ThisType(TypeRef(NoPrefix,module class scala)),Product2))

[log frontend]             <== collectCompanions(Product2[(Tuple2.this.T1, Tuple2.this.T2)(Tuple2.this), (Tuple2.this.T1, Tuple2.this.T2)(Tuple2.this)]) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(ThisType(TypeRef(NoPrefix,module class scala)),Product2))

[log frontend]           <== collectCompanions(Tuple2) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(ThisType(TypeRef(NoPrefix,module class scala)),Tuple2), TermRef(ThisType(TypeRef(NoPrefix,module class scala)),Product2), TermRef(ThisType(TypeRef(NoPrefix,module class io)),Serializable))

[log frontend]         <== collectCompanions(scala.collection.Iterable[(scala.collection.Map[Map.this.K, Map.this.V](Map.this), scala.collection.Map[Map.this.K, Map.this.V](Map.this))]) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class scala)),collection),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(ThisType(TypeRef(NoPrefix,module class scala)),Tuple2), TermRef(ThisType(TypeRef(NoPrefix,module class scala)),Product2), TermRef(ThisType(TypeRef(NoPrefix,module class io)),Serializable))

Notice that Iterable itself is not in that list! Compare with what we get when I remove some files from i3250.sh and the compilation succeeds:

[log frontend]           ==> collectCompanions(scala.collection.Iterable[(scala.collection.Map[Map.this.K, Map.this.V](Map.this), scala.collection.Map[Map.this.K, Map.this.V](Map.this))])?

[log frontend]             ==> collectCompanions(scala.collection.Iterable)?

[log frontend]               ==> collectCompanions(scala.collection.generic.GenericTraversableTemplate[scala.collection.Iterable[Iterable.this.A](Iterable.this), scala.collection.Iterable[A]])?

[log frontend]                 ==> collectCompanions(scala.collection.generic.GenericTraversableTemplate)?

[log frontend]                   ==> collectCompanions(scala.collection.generic.HasNewBuilder[scala.collection.generic.GenericTraversableTemplate[GenericTraversableTemplate.this.A, GenericTraversableTemplate.this.CC](GenericTraversableTemplate.this), 
  (scala.collection.generic.GenericTraversableTemplate[GenericTraversableTemplate.this.A, GenericTraversableTemplate.this.CC](GenericTraversableTemplate.this) & scala.collection.GenTraversable & 
    GenericTraversableTemplate.this.A
  ) @uncheckedVariance
])?

[log frontend]                     ==> collectCompanions(scala.collection.GenTraversable)?

[log frontend]                       ==> collectCompanions(scala.collection.generic.GenericTraversableTemplate[scala.collection.GenTraversable[GenTraversable.this.A](GenTraversable.this), scala.collection.GenTraversable[A]])?

[log frontend]                       <== collectCompanions(scala.collection.generic.GenericTraversableTemplate[scala.collection.GenTraversable[GenTraversable.this.A](GenTraversable.this), scala.collection.GenTraversable[A]]) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),generic),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class scala)),collection),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package))

[log frontend]                     <== collectCompanions(scala.collection.GenTraversable) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class scala)),collection),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),GenTraversable), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),generic),package))

[log frontend]                   <== collectCompanions(scala.collection.generic.HasNewBuilder[scala.collection.generic.GenericTraversableTemplate[GenericTraversableTemplate.this.A, GenericTraversableTemplate.this.CC](GenericTraversableTemplate.this), 
  (scala.collection.generic.GenericTraversableTemplate[GenericTraversableTemplate.this.A, GenericTraversableTemplate.this.CC](GenericTraversableTemplate.this) & scala.collection.GenTraversable & 
    GenericTraversableTemplate.this.A
  ) @uncheckedVariance
]) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),generic),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class scala)),collection),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),GenTraversable))

[log frontend]                 <== collectCompanions(scala.collection.generic.GenericTraversableTemplate) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),generic),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class scala)),collection),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),GenTraversable))

[log frontend]               <== collectCompanions(scala.collection.generic.GenericTraversableTemplate[scala.collection.Iterable[Iterable.this.A](Iterable.this), scala.collection.Iterable[A]]) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),generic),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class scala)),collection),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),GenTraversable))

[log frontend]             <== collectCompanions(scala.collection.Iterable) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class scala)),collection),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),Iterable), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),Traversable), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),generic),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),TraversableOnce), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),parallel),package), TermRef(ThisType(TypeRef(NoPrefix,module class parallel)),ParIterable), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),GenIterable), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),GenTraversable))

[log frontend]           <== collectCompanions(scala.collection.Iterable[(scala.collection.Map[Map.this.K, Map.this.V](Map.this), scala.collection.Map[Map.this.K, Map.this.V](Map.this))]) = TermRefSet(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class scala)),collection),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),Iterable), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),scala),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),Traversable), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),generic),package), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),TraversableOnce), TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),parallel),package), TermRef(ThisType(TypeRef(NoPrefix,module class parallel)),ParIterable), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),GenIterable), TermRef(ThisType(TypeRef(NoPrefix,module class collection)),GenTraversable), TermRef(ThisType(TypeRef(NoPrefix,module class scala)),Tuple2), TermRef(ThisType(TypeRef(NoPrefix,module class scala)),Product2), TermRef(ThisType(TypeRef(NoPrefix,module class io)),Serializable))

[smarter]

Hmm, I might have been confused by recursive collectCompanions calls, but there's still something weird going on where Iterable is not present when it should be in the implicit search done right before the error, contrast:

AmbiguousImplicits: https://gist.github.com/smarter/d0bee25acd16a8859723918c875b3dc8
SearchSuccess: https://gist.github.com/smarter/bfe7c34d817ab7809516fa375aa51049

If you look at the outermost collectCompanions (so, no partial results due to having searched some companions before), the successful one contains more stuff, including TermRef(ThisType(TypeRef(NoPrefix,module class collection)),Iterable), compared to the failed one.

[smarter]

The issue goes away if Config.cacheImplicitScopes is set to false.

[smarter]

Reopening to see if we can find the underlying cause even though we have a workaround now.