Port swiftSyntax to Windows #7588

hughbe · 2017-02-18T06:32:48Z

strndup is not available on Windows. Windows does provide strdup but that isn't the behaviour we want unfortunately.

So roll out our own solution

@bitjammer

bitjammer

To add a test:

Add an OwnedString.cpp to ./unittests
- Don't forget the license header; see other files in the directory.
Include OwnedString.cpp in ./unittests/CMakeLists.txt

bitjammer · 2017-02-18T06:41:22Z

include/swift/Basic/OwnedString.h

+
+  return data;
+}
+


Let's do the following, please.

Change OwnedString(const char *Data, size_t Length, StringOwnership Ownership) to:
OwnedString(const char *Data, size_t Length)

assume StringOwnership::Unowned

Move copyString(const char *s, size_t length) to OwnedString OwnedString::copy()

Add rigorous testing for copying. This is an absolute must if we are going to be implementing the equivalent to strndup.

gottesmm · 2017-02-18T07:08:21Z

@bitjammer It is also important that the test follows the naming convention. Specifically, it should be OwnedStringTest.cpp like all of the other tests in ./unittests.

hughbe · 2017-02-20T03:24:07Z

Okay, so I'm getting these errors "pointer being freed was not allocated". Odd, considering the only time we are StringOwnership::Copied is when we call OwnedString::copy. This always causes an allocation

I'm not really sure where I'm going wrong - If you guys have any ideas, I'd appreciate it!

bitjammer · 2017-02-20T09:16:54Z

You took the copy out of OwnedString(const OwnedString &Other), but you're still reusing the StringOwnership argument. You need to do a copy there in that constructor if the ownership is Copied, otherwise you'll free the same pointer twice.

hughbe · 2017-02-20T10:55:14Z

Thanks David - you're right! I've updated the PR and added some unit tests that pass on Windows. I haven't got anything in tests/* working on Windows yet, so let's see!

hughbe · 2017-02-20T11:13:36Z

@swift-ci please smoke test

jrose-apple · 2017-02-20T17:10:37Z

include/swift/Basic/OwnedString.h

@@ -88,12 +103,13 @@ class OwnedString {
  }

  ~OwnedString() {
-    if (Ownership == StringOwnership::Copied)
+    if (Ownership == StringOwnership::Copied && Data) {


This shouldn't be necessary; free accepts a null pointer.

hughbe · 2017-02-21T01:48:09Z

@swift-ci please smoke test

hughbe · 2017-02-21T01:48:36Z

@swift-ci please smoke test

bitjammer · 2017-02-21T03:01:25Z

@swift-ci Please asan test

bitjammer · 2017-02-21T03:14:19Z

It looks like this had been kicked off: https://ci.swift.org/view/All/job/swift-PR-osx-ASAN-test/47/

bitjammer · 2017-02-21T03:14:38Z

(But no progress reported)

shahmishal · 2017-02-21T03:16:25Z

its due to wrong sha

Setting status of d0eed9e97782d1b30b6481341a92f4a1b045510a to PENDING with url https://ci.swift.org/job/swift-PR-osx-ASAN-test/47/ and message: 'Build started.'

shahmishal · 2017-02-21T03:16:38Z

@swift-ci Please asan test

shahmishal · 2017-02-21T03:17:20Z

Setting status of b564a91 to PENDING with url https://ci.swift.org/job/swift-PR-osx-ASAN-test/48/ and message: 'Build started.'

This worked now, sorry for the trouble.

hughbe · 2017-02-21T06:21:44Z

FAIL: Swift(macosx-x86_64) :: Demangle/remangle.swift (8764 of 9405)
******************** TEST 'Swift(macosx-x86_64) :: Demangle/remangle.swift' FAILED ********************
Script:
--
sed -ne '/--->/s/ *--->.*$//p' < /Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/swift/test/Demangle/Inputs/manglings.txt > /Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/test-macosx-x86_64/Demangle/Output/remangle.swift.tmp.input
swift-demangle -test-remangle < /Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/test-macosx-x86_64/Demangle/Output/remangle.swift.tmp.input > /Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/test-macosx-x86_64/Demangle/Output/remangle.swift.tmp.output
diff /Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/test-macosx-x86_64/Demangle/Output/remangle.swift.tmp.input /Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/test-macosx-x86_64/Demangle/Output/remangle.swift.tmp.output
--
Exit Code: 134

Command Output (stderr):
--
=================================================================
==42010==ERROR: AddressSanitizer: heap-use-after-free on address 0x607000060892 at pc 0x00010390f689 bp 0x7fff5c3965b0 sp 0x7fff5c3965a8
READ of size 1 at 0x607000060892 thread T0
atos[42011]: [fatal] unable to ask for permission to examine process; run tool using sudo, or without redirecting stdin and stderr.
==42010==WARNING: Can't read from symbolizer at fd 5
atos[42012]: [fatal] unable to ask for permission to examine process; run tool using sudo, or without redirecting stdin and stderr.
==42010==WARNING: Can't read from symbolizer at fd 5
atos[42013]: [fatal] unable to ask for permission to examine process; run tool using sudo, or without redirecting stdin and stderr.
==42010==WARNING: Can't read from symbolizer at fd 5
atos[42014]: [fatal] unable to ask for permission to examine process; run tool using sudo, or without redirecting stdin and stderr.
==42010==WARNING: Can't read from symbolizer at fd 5
==42010==WARNING: Failed to use and restart external symbolizer!
    #0 0x10390f688 in (anonymous namespace)::Remangler::trySubstitution(swift::Demangle::Node*, (anonymous namespace)::SubstitutionEntry&, bool) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x1000a7688)
    #1 0x1039148fa in (anonymous namespace)::Remangler::mangleIdentifierImpl(swift::Demangle::Node*, bool) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x1000ac8fa)
    #2 0x1038feae5 in (anonymous namespace)::Remangler::mangle(swift::Demangle::Node*) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x100096ae5)
    #3 0x10390425d in (anonymous namespace)::Remangler::mangle(swift::Demangle::Node*) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x10009c25d)
    #4 0x1038fd18d in swift::Demangle::mangleNode(std::__1::shared_ptr<swift::Demangle::Node> const&) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x10009518d)
    #5 0x10386ba13 in demangle(llvm::raw_ostream&, llvm::StringRef, swift::Demangle::Context&, swift::Demangle::DemangleOptions const&) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x100003a13)
    #6 0x10386aa7b in main (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x100002a7b)
    #7 0x7fffcd5b1254 in start (/usr/lib/system/libdyld.dylib:x86_64+0x5254)

0x607000060892 is located 18 bytes inside of 72-byte region [0x607000060880,0x6070000608c8)
freed by thread T0 here:
    #0 0x103c03cab in wrap__ZdlPv (/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/8.1.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x61cab)
    #1 0x7fffcc183dad in std::__1::__shared_weak_count::__release_shared() (/usr/lib/libc++.1.dylib:x86_64+0x3bdad)
    #2 0x1038feb00 in (anonymous namespace)::Remangler::mangle(swift::Demangle::Node*) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x100096b00)
    #3 0x10390425d in (anonymous namespace)::Remangler::mangle(swift::Demangle::Node*) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x10009c25d)
    #4 0x1038fd18d in swift::Demangle::mangleNode(std::__1::shared_ptr<swift::Demangle::Node> const&) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x10009518d)
    #5 0x10386ba13 in demangle(llvm::raw_ostream&, llvm::StringRef, swift::Demangle::Context&, swift::Demangle::DemangleOptions const&) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x100003a13)
    #6 0x10386aa7b in main (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x100002a7b)
    #7 0x7fffcd5b1254 in start (/usr/lib/system/libdyld.dylib:x86_64+0x5254)

previously allocated by thread T0 here:
    #0 0x103c036ab in wrap__Znwm (/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/8.1.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x616ab)
    #1 0x1038fe99f in (anonymous namespace)::Remangler::mangle(swift::Demangle::Node*) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x10009699f)
    #2 0x10390425d in (anonymous namespace)::Remangler::mangle(swift::Demangle::Node*) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x10009c25d)
    #3 0x1038fd18d in swift::Demangle::mangleNode(std::__1::shared_ptr<swift::Demangle::Node> const&) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x10009518d)
    #4 0x10386ba13 in demangle(llvm::raw_ostream&, llvm::StringRef, swift::Demangle::Context&, swift::Demangle::DemangleOptions const&) (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x100003a13)
    #5 0x10386aa7b in main (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x100002a7b)
    #6 0x7fffcd5b1254 in start (/usr/lib/system/libdyld.dylib:x86_64+0x5254)

SUMMARY: AddressSanitizer: heap-use-after-free (/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/bin/swift-demangle:x86_64+0x1000a7688) in (anonymous namespace)::Remangler::trySubstitution(swift::Demangle::Node*, (anonymous namespace)::SubstitutionEntry&, bool)
Shadow bytes around the buggy address:
  0x1c0e0000c0c0: 00 00 00 00 01 fa fa fa fa fa 00 00 00 00 00 00
  0x1c0e0000c0d0: 00 00 01 fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x1c0e0000c0e0: 00 00 fa fa fa fa 00 00 00 00 00 00 00 00 01 fa
  0x1c0e0000c0f0: fa fa fa fa 00 00 00 00 00 00 00 00 01 fa fa fa
  0x1c0e0000c100: fa fa 00 00 00 00 00 00 00 00 00 fa fa fa fa fa
=>0x1c0e0000c110: fd fd[fd]fd fd fd fd fd fd fa fa fa fa fa 00 00
  0x1c0e0000c120: 00 00 00 00 00 00 00 fa fa fa fa fa 00 00 00 00
  0x1c0e0000c130: 00 00 00 00 00 fa fa fa fa fa 00 00 00 00 00 00
  0x1c0e0000c140: 00 00 00 fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x1c0e0000c150: 00 fa fa fa fa fa 00 00 00 00 00 00 00 00 00 fa
  0x1c0e0000c160: fa fa fa fa 00 00 00 00 00 00 00 00 00 fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==42010==ABORTING
/Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/test-macosx-x86_64/Demangle/Output/remangle.swift.script: line 3: 42010 Abort trap: 6           swift-demangle -test-remangle < /Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/test-macosx-x86_64/Demangle/Output/remangle.swift.tmp.input > /Users/buildnode/jenkins/workspace/swift-PR-osx-ASAN-test/buildbot_incremental_asan/swift-macosx-x86_64/test-macosx-x86_64/Demangle/Output/remangle.swift.tmp.output

--

********************

hughbe · 2017-02-21T06:22:07Z

Is the ASAN failure related?

hughbe · 2017-02-21T08:53:57Z

#7664

shahmishal · 2017-02-21T18:46:00Z

This should be fixed now.

@swift-ci Please asan test

shahmishal · 2017-02-21T19:46:21Z

I will re-trigger this asan job right after I update the Xcode.

shahmishal · 2017-02-21T21:00:18Z

@swift-ci Please asan test

hughbe · 2017-02-22T02:51:13Z

Great! Thanks Mishal. Tests have passed, let me know if you have any more comments

bitjammer

Looks good - thanks so much for taking the time to do this right. I appreciate it.

bitjammer requested changes Feb 18, 2017

View reviewed changes

jrose-apple reviewed Feb 20, 2017

View reviewed changes

Port swiftSyntax to Windows

b564a91

hughbe mentioned this pull request Feb 22, 2017

Add support for building Swift on Windows with clang-cl and MSVC #5904

Merged

bitjammer approved these changes Feb 22, 2017

View reviewed changes

hughbe merged commit 32260e6 into swiftlang:master Feb 22, 2017

hughbe deleted the swiftSyntax-win32 branch February 22, 2017 03:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port swiftSyntax to Windows #7588

Port swiftSyntax to Windows #7588

hughbe commented Feb 18, 2017

bitjammer left a comment

bitjammer Feb 18, 2017

gottesmm commented Feb 18, 2017

hughbe commented Feb 20, 2017 •

edited

Loading

bitjammer commented Feb 20, 2017

hughbe commented Feb 20, 2017

hughbe commented Feb 20, 2017

jrose-apple Feb 20, 2017

hughbe commented Feb 21, 2017

hughbe commented Feb 21, 2017

bitjammer commented Feb 21, 2017

bitjammer commented Feb 21, 2017

bitjammer commented Feb 21, 2017

shahmishal commented Feb 21, 2017

shahmishal commented Feb 21, 2017

shahmishal commented Feb 21, 2017

hughbe commented Feb 21, 2017

hughbe commented Feb 21, 2017

hughbe commented Feb 21, 2017

shahmishal commented Feb 21, 2017

shahmishal commented Feb 21, 2017

shahmishal commented Feb 21, 2017

hughbe commented Feb 22, 2017

bitjammer left a comment

Port swiftSyntax to Windows #7588

Port swiftSyntax to Windows #7588

Conversation

hughbe commented Feb 18, 2017

bitjammer left a comment

Choose a reason for hiding this comment

bitjammer Feb 18, 2017

Choose a reason for hiding this comment

gottesmm commented Feb 18, 2017

hughbe commented Feb 20, 2017 • edited Loading

bitjammer commented Feb 20, 2017

hughbe commented Feb 20, 2017

hughbe commented Feb 20, 2017

jrose-apple Feb 20, 2017

Choose a reason for hiding this comment

hughbe commented Feb 21, 2017

hughbe commented Feb 21, 2017

bitjammer commented Feb 21, 2017

bitjammer commented Feb 21, 2017

bitjammer commented Feb 21, 2017

shahmishal commented Feb 21, 2017

shahmishal commented Feb 21, 2017

shahmishal commented Feb 21, 2017

hughbe commented Feb 21, 2017

hughbe commented Feb 21, 2017

hughbe commented Feb 21, 2017

shahmishal commented Feb 21, 2017

shahmishal commented Feb 21, 2017

shahmishal commented Feb 21, 2017

hughbe commented Feb 22, 2017

bitjammer left a comment

Choose a reason for hiding this comment

hughbe commented Feb 20, 2017 •

edited

Loading