Skip to content

Merge 1.0.x to master #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Oct 27, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
b8b0f87
Speed up line/column in OffsetPosition
liskin Jul 3, 2015
f809b06
Fix packrat caching with PagedSeqReader
liskin Aug 4, 2015
b6ee1f6
Use a thread local WeakHashMap instead of synchronized
liskin Aug 6, 2015
0345e49
Add test for #45
liskin Aug 8, 2015
895a882
Merge pull request #65 from liskin/pagedseqreader-packrat
gourlaysama Sep 8, 2015
aa52548
[backport] add maintenance status on README, remove link to JIRA
gourlaysama Jul 27, 2015
29960ad
Merge pull request #67 from gourlaysama/readme
gourlaysama Sep 9, 2015
ab6e080
Fix binary incompatibilities in object OffsetPosition
liskin Sep 11, 2015
4600c5a
Merge pull request #68 from liskin/offsetposition-index-cache
gourlaysama Sep 11, 2015
057730a
Fix whitespace in Parsers.phrase
liskin Sep 15, 2015
72ea0d7
Fix lastNoSuccessVar memory leak
liskin Sep 15, 2015
01c5bac
Add test for SI-9010
liskin Sep 15, 2015
ee2708e
Merge pull request #69 from liskin/lastnosuccess-memleak
gourlaysama Feb 10, 2016
3eef456
Fixes OffsetPosition.lineContents so that it doesn't grab a newline a…
toddobryan-amazon Dec 29, 2015
51d69b4
Add Travis workaround based on both hostname and /etc/hosts
toddobryan-amazon Jan 6, 2016
845d5ab
Merge pull request #77 from gourlaysama/issues/56-offsetposition-line…
gourlaysama Feb 10, 2016
26d36ac
bump scala, sbt and scala-module-plugin versions
gourlaysama Feb 12, 2016
4d9cb2c
Merge pull request #79 from gourlaysama/bump2
gourlaysama Feb 13, 2016
3dd55dd
Change apostrophe in error message
egulias Jul 19, 2016
ac7c33f
fix travis-ci hosts
egulias Sep 9, 2016
6582a95
Merge pull request #81 from egulias/patch-1
gourlaysama Sep 19, 2016
ad63597
[backport] Scala.js support for 1.0.x
gourlaysama Sep 20, 2016
151ef0d
Merge pull request #83 from gourlaysama/backports1
gourlaysama Sep 21, 2016
5df662c
bump sbt, scala & dependency versions to latest
gourlaysama Sep 20, 2016
0b01f43
Merge pull request #84 from gourlaysama/bump3
gourlaysama Sep 22, 2016
cdfba62
Redo how cross-version publishing works
gourlaysama Oct 4, 2016
b71f12e
Merge pull request #86 from gourlaysama/publishing
SethTisue Oct 6, 2016
f65fbe7
Merge branch '1.0.x' into merge-1.0.x-master
gourlaysama Oct 8, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@ sudo: false

language: scala

addons:
hosts:
- myshorthost
hostname: myshorthost

env:
global:
- PUBLISH_JDK=openjdk6
# PGP_PASSPHRASE
- secure: "SkBtn/6OjEldoikn0MFuyeLT/pau27kwKSDYTVQeJ4BKDzdWLwLE5Q3RukLGttIfNdhOvRoocpQSW9GkZfibTHmwrRnAokucfZCqTsKbwoOp1xIoOh5GrrVrB6gcP7WBTKinqFdBgSvLOrP7GviImz4ZuB9wq1r+mToGG4pDrXc="
# SONA_USER
Expand All @@ -18,7 +22,6 @@ script: admin/build.sh

jdk:
- openjdk6
- openjdk7
- oraclejdk8

notifications:
Expand Down
3 changes: 1 addition & 2 deletions admin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ To configure tag driven releases from Travis CI.
Edit `.travis.yml` as prompted.
4. Edit `.travis.yml` to use `./admin/build.sh` as the build script,
and edit that script to use the tasks required for this project.
5. Edit `.travis.yml` to select which JDK will be used for publishing.
5. Edit `build.sbt` to select which JDK will be used for publishing.

It is important to add comments in .travis.yml to identify the name
of each environment variable encoded in a `:secure` section.
Expand All @@ -30,7 +30,6 @@ form:
language: scala
env:
global:
- PUBLISH_JDK=openjdk6
# PGP_PASSPHRASE
- secure: "XXXXXX"
# SONA_USER
Expand Down
2 changes: 1 addition & 1 deletion admin/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ set -e
# git on travis does not fetch tags, but we have TRAVIS_TAG
# headTag=$(git describe --exact-match ||:)

if [ "$TRAVIS_JDK_VERSION" == "$PUBLISH_JDK" ] && [[ "$TRAVIS_TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9-]+)? ]]; then
if [[ "$TRAVIS_TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9-]+)? ]]; then
echo "Going to release from tag $TRAVIS_TAG!"
myVer=$(echo $TRAVIS_TAG | sed -e s/^v//)
publishVersion='set every version := "'$myVer'"'
Expand Down
35 changes: 25 additions & 10 deletions build.sbt
Original file line number Diff line number Diff line change
@@ -1,19 +1,34 @@
scalaVersion in ThisBuild := crossScalaVersions.value.head

crossScalaVersions in ThisBuild := {
val v211 = List("2.11.8")
val v212 = List("2.12.0-RC1")

val javaVersion = System.getProperty("java.version")
val isJDK6Or7 =
javaVersion.startsWith("1.6.") || javaVersion.startsWith("1.7.")
if (isJDK6Or7)
Seq("2.11.7")
else
Seq("2.11.7", "2.12.0-M3")
val isTravisPublishing = !util.Properties.envOrElse("TRAVIS_TAG", "").trim.isEmpty

if (isTravisPublishing) {
if (javaVersion.startsWith("1.6.")) v211
else if (javaVersion.startsWith("1.8.")) v212
else Nil
} else if (javaVersion.startsWith("1.6.") || javaVersion.startsWith("1.7.")) {
v211
} else if (javaVersion.startsWith("1.8.") || javaVersion.startsWith("9")) {
v211 ++ v212
} else {
sys.error(s"Unsupported java version: $javaVersion.")
}
}

lazy val `scala-parser-combinators` = crossProject.in(file(".")).
settings(scalaModuleSettings: _*).
settings(
name := "scala-parser-combinators-root"
).
jvmSettings(
name := "scala-parser-combinators-jvm"
// Mima uses the name of the jvm project in the artifactId
// when resolving previous versions (so no "-jvm" project)
name := "scala-parser-combinators"
).
jsSettings(
name := "scala-parser-combinators-js"
Expand All @@ -38,10 +53,10 @@ lazy val `scala-parser-combinators` = crossProject.in(file(".")).
).
jsConfigure(_.enablePlugins(ScalaJSJUnitPlugin)).
jvmSettings(
libraryDependencies += "junit" % "junit" % "4.11" % "test",
libraryDependencies += "com.novocode" % "junit-interface" % "0.10" % "test"
libraryDependencies += "junit" % "junit" % "4.12" % "test",
libraryDependencies += "com.novocode" % "junit-interface" % "0.11" % "test"
).
settings(
jvmSettings(
mimaPreviousVersion := None
)

Expand Down
14 changes: 14 additions & 0 deletions js/src/main/scala/scala/util/parsing/input/PositionCache.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package scala.util.parsing.input

import java.lang.CharSequence
import java.util.{AbstractMap, Collections}

private[input] trait PositionCache {
private[input] lazy val indexCache: java.util.Map[CharSequence,Array[Int]] = new AbstractMap[CharSequence, Array[Int]] {

override def entrySet() = Collections.emptySet()

// the /dev/null of Maps
override def put(ch: CharSequence, a: Array[Int]) = null
}
}
17 changes: 17 additions & 0 deletions jvm/src/main/scala/scala/util/parsing/input/PositionCache.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package scala.util.parsing.input

import java.lang.{CharSequence, ThreadLocal}
import java.util.WeakHashMap

/**
* @author Tomáš Janoušek
*/
private[input] trait PositionCache {
private lazy val indexCacheTL =
// not DynamicVariable as that would share the map from parent to child :-(
new ThreadLocal[java.util.Map[CharSequence, Array[Int]]] {
override def initialValue = new WeakHashMap[CharSequence, Array[Int]]
}

private[input] def indexCache = indexCacheTL.get
}
51 changes: 51 additions & 0 deletions jvm/src/test/scala/scala/util/parsing/combinator/t9010.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import scala.util.parsing.combinator._
import scala.util.DynamicVariable

import org.junit.Test

class t9010 {
@Test
def test: Unit = {
val p = new grammar
val lastNoSuccessVar = getLastNoSuccessVar(p)
import p._

val res1 = parse(x, "x")
assert(res1.successful)
assert(lastNoSuccessVar.value == None)

val res2 = parse(x, "y")
assert(!res2.successful)
assert(lastNoSuccessVar.value == None)

val res3 = parseAll(x, "x")
assert(res3.successful)
assert(lastNoSuccessVar.value == None)

val res4 = parseAll(x, "y")
assert(!res4.successful)
assert(lastNoSuccessVar.value == None)
}

private def getLastNoSuccessVar(p: Parsers): DynamicVariable[Option[_]] = {
// use java reflection instead of scala (see below) because of
// https://issues.scala-lang.org/browse/SI-9306
val fn = "scala$util$parsing$combinator$Parsers$$lastNoSuccessVar"
val f = p.getClass.getDeclaredMethod(fn)
f.setAccessible(true)
f.invoke(p).asInstanceOf[DynamicVariable[Option[_]]]

/*
val ru = scala.reflect.runtime.universe
val mirror = ru.runtimeMirror(getClass.getClassLoader)
val lastNoSuccessVarField =
ru.typeOf[Parsers].decl(ru.TermName("lastNoSuccessVar")).asTerm.accessed.asTerm
mirror.reflect(p).reflectField(lastNoSuccessVarField).get.
asInstanceOf[DynamicVariable[Option[_]]]
*/
}

private final class grammar extends RegexParsers {
val x: Parser[String] = "x"
}
}
2 changes: 1 addition & 1 deletion project/build.properties
Original file line number Diff line number Diff line change
@@ -1 +1 @@
sbt.version=0.13.8
sbt.version=0.13.12
4 changes: 2 additions & 2 deletions project/plugins.sbt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
addSbtPlugin("org.scala-lang.modules" % "scala-module-plugin" % "1.0.3")
addSbtPlugin("org.scala-lang.modules" % "scala-module-plugin" % "1.0.4")

addSbtPlugin("org.scala-js" % "sbt-scalajs" % "0.6.6")
addSbtPlugin("org.scala-js" % "sbt-scalajs" % "0.6.12")
27 changes: 16 additions & 11 deletions shared/src/main/scala/scala/util/parsing/combinator/Parsers.scala
Original file line number Diff line number Diff line change
Expand Up @@ -156,14 +156,19 @@ trait Parsers {
val successful = true
}

private lazy val lastNoSuccessVar = new DynamicVariable[Option[NoSuccess]](None)
/* two layers of Option:
* outer Option is None if lastNoSuccess tracking is disabled (outside of
* phrase) and Some if tracking is enabled
* inner Option is None if NoSuccess hasn't been seen yet, Some otherwise
* this is necessary to avoid leaking NoSuccesses in thread locals */
private lazy val lastNoSuccessVar = new DynamicVariable[Option[Option[NoSuccess]]](None)

/** A common super-class for unsuccessful parse results. */
sealed abstract class NoSuccess(val msg: String, override val next: Input) extends ParseResult[Nothing] { // when we don't care about the difference between Failure and Error
val successful = false

if (lastNoSuccessVar.value forall (v => !(next.pos < v.next.pos)))
lastNoSuccessVar.value = Some(this)
if (lastNoSuccessVar.value exists (_ forall (v => !(next.pos < v.next.pos))))
lastNoSuccessVar.value = Some(Some(this))

def map[U](f: Nothing => U) = this
def mapPartial[U](f: PartialFunction[Nothing, U], error: Nothing => String): ParseResult[U] = this
Expand Down Expand Up @@ -590,7 +595,7 @@ trait Parsers {
* @return a `tParser` that succeeds if `e` is the next available input.
*/

implicit def accept(e: Elem): Parser[Elem] = acceptIf(_ == e)("`"+e+"' expected but " + _ + " found")
implicit def accept(e: Elem): Parser[Elem] = acceptIf(_ == e)("'"+e+"' expected but " + _ + " found")

/** A parser that matches only the given list of element `es`.
*
Expand Down Expand Up @@ -908,14 +913,14 @@ trait Parsers {
* if `p` consumed all the input.
*/
def phrase[T](p: Parser[T]) = new Parser[T] {
def apply(in: Input) = lastNoSuccessVar.withValue(None) {
def apply(in: Input) = lastNoSuccessVar.withValue(Some(None)) {
p(in) match {
case s @ Success(out, in1) =>
if (in1.atEnd)
s
else
lastNoSuccessVar.value filterNot { _.next.pos < in1.pos } getOrElse Failure("end of input expected", in1)
case ns => lastNoSuccessVar.value.getOrElse(ns)
case s @ Success(out, in1) =>
if (in1.atEnd)
s
else
lastNoSuccessVar.value flatMap (_ filterNot { _.next.pos < in1.pos }) getOrElse Failure("end of input expected", in1)
case ns => lastNoSuccessVar.value.flatten.getOrElse(ns)
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ package scala
package util.parsing.input

import scala.collection.mutable.ArrayBuffer
import java.lang.{CharSequence, ThreadLocal}
import java.util.WeakHashMap

/** `OffsetPosition` is a standard class for positions
* represented as offsets into a source ``document''.
Expand All @@ -19,10 +21,20 @@ import scala.collection.mutable.ArrayBuffer
*
* @author Martin Odersky
*/
case class OffsetPosition(source: java.lang.CharSequence, offset: Int) extends Position {
case class OffsetPosition(source: CharSequence, offset: Int) extends Position {

/** An index that contains all line starts, including first line, and eof. */
private lazy val index: Array[Int] = {
Option(OffsetPosition.indexCache.get(source)) match {
case Some(index) => index
case None =>
val index = genIndex
OffsetPosition.indexCache.put(source, index)
index
}
}

private def genIndex: Array[Int] = {
val lineStarts = new ArrayBuffer[Int]
lineStarts += 0
for (i <- 0 until source.length)
Expand Down Expand Up @@ -50,8 +62,14 @@ case class OffsetPosition(source: java.lang.CharSequence, offset: Int) extends P
*
* @return the line at `offset` (not including a newline)
*/
def lineContents: String =
source.subSequence(index(line - 1), index(line)).toString
def lineContents: String = {
val endIndex = if (source.charAt(index(line) - 1) == '\n') {
index(line) - 1
} else {
index(line)
}
source.subSequence(index(line - 1), endIndex).toString
}

/** Returns a string representation of the `Position`, of the form `line.column`. */
override def toString = line+"."+column
Expand All @@ -71,3 +89,7 @@ case class OffsetPosition(source: java.lang.CharSequence, offset: Int) extends P
this.line == that.line && this.column < that.column
}
}

/** An object holding the index cache.
*/
object OffsetPosition extends scala.runtime.AbstractFunction2[CharSequence,Int,OffsetPosition] with PositionCache
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,10 @@ object PagedSeqReader {
* @author Martin Odersky
*/
class PagedSeqReader(seq: PagedSeq[Char],
override val offset: Int) extends Reader[Char] {
override val offset: Int) extends Reader[Char] { outer =>
import PagedSeqReader._

override lazy val source: java.lang.CharSequence = seq
override val source: java.lang.CharSequence = seq

/** Construct a `PagedSeqReader` with its first element at
* `source(0)` and position `(1,1)`.
Expand All @@ -51,7 +51,9 @@ class PagedSeqReader(seq: PagedSeq[Char],
* otherwise, it's a `PagedSeqReader` containing the rest of input.
*/
def rest: PagedSeqReader =
if (seq.isDefinedAt(offset)) new PagedSeqReader(seq, offset + 1)
if (seq.isDefinedAt(offset)) new PagedSeqReader(seq, offset + 1) {
override val source: java.lang.CharSequence = outer.source
}
else this

/** The position of the first element in the reader.
Expand All @@ -67,5 +69,7 @@ class PagedSeqReader(seq: PagedSeq[Char],
* `n` elements.
*/
override def drop(n: Int): PagedSeqReader =
new PagedSeqReader(seq, offset + n)
new PagedSeqReader(seq, offset + n) {
override val source: java.lang.CharSequence = outer.source
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ class PackratParsersTest {
val failure = parseResult.asInstanceOf[Failure]
assertEquals(expectedFailureMsg, failure.msg)
}
assertFailure("``b'' expected but `c' found", "a a a a b b b c c c c")
assertFailure("'`b'' expected but `c' found", "a a a a b b b c c c c")
assertFailure("end of input", "a a a a b b b b c c c")
}

Expand Down
46 changes: 46 additions & 0 deletions shared/src/test/scala/scala/util/parsing/combinator/gh45.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package scala.util.parsing.combinator

import scala.util.parsing.input._
import scala.collection.immutable.PagedSeq

import org.junit.Test
import org.junit.Assert.assertTrue

import scala.util.parsing.combinator.syntactical.StandardTokenParsers

class gh45 {

@Test
def test4: Unit = {
def check(rd: Reader[Char]): Unit = {
val g = new grammar
val p = g.phrase(g.script)
val parseResult = p(new g.lexical.Scanner(rd))
assertTrue(parseResult.isInstanceOf[g.Success[_]])
}

val str = "x once y"
check(new CharSequenceReader(str))
/* Note that this only tests PagedSeq.rest since neither
* PackratReader nor lexical.Scanner override/use the drop method.
*/
check(new PagedSeqReader(PagedSeq.fromStrings(List(str))))
}

}

private final class grammar extends StandardTokenParsers with PackratParsers {
lexical.reserved ++= List("x", "y", "z", "once")

var onceCnt: Int = 0
lazy val once: PackratParser[String] = memo("once") ^? {
case s if onceCnt == 0 =>
onceCnt += 1
s
}

lazy val script: PackratParser[Any] =
( "x" ~ once ~ "z"
| "x" ~ once ~ "y"
)
}
Loading