[libclang] Always Dup in createRef(StringRef) #125020

vitalybuka · 2025-01-30T03:59:32Z

We can't guaranty that underlying string is
0-terminated and [String.size()] is even in the
same allocation.

https://lab.llvm.org/buildbot/#/builders/94/builds/4152/steps/17/logs/stdio

==c-index-test==1846256==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0  in clang::cxstring::createRef(llvm::StringRef) llvm-project/clang/tools/libclang/CXString.cpp:96:36
    #1  in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:521:39
    #2  in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7
    #3  in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7
    #4  in DumpCXComment llvm-project/clang/tools/c-index-test/c-index-test.c:685:3
    #5  in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:768:7

  Memory was marked as uninitialized
    #0  in __msan_allocated_memory llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:1023:5
    #1  in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:172:7
    #2  in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:216:12
    #3  in Allocate llvm-project/llvm/include/llvm/Support/AllocatorBase.h:53:43
    #4  in Allocate<char> llvm-project/llvm/include/llvm/Support/AllocatorBase.h:76:29
    #5  in convertCodePointToUTF8 llvm-project/clang/lib/AST/CommentLexer.cpp:42:30
    #6  in clang::comments::Lexer::resolveHTMLDecimalCharacterReference(llvm::StringRef) const llvm-project/clang/lib/AST/CommentLexer.cpp:76:10
    #7  in clang::comments::Lexer::lexHTMLCharacterReference(clang::comments::Token&) llvm-project/clang/lib/AST/CommentLexer.cpp:615:16
    #8  in consumeToken llvm-project/clang/include/clang/AST/CommentParser.h:62:9
    #9  in clang::comments::Parser::parseParagraphOrBlockCommand() llvm-project/clang/lib/AST/CommentParser.cpp
    #10 in clang::comments::Parser::parseFullComment() llvm-project/clang/lib/AST/CommentParser.cpp:925:22
    #11 in clang::RawComment::parse(clang::ASTContext const&, clang::Preprocessor const*, clang::Decl const*) const llvm-project/clang/lib/AST/RawCommentList.cpp:221:12
    #12 in clang::ASTContext::getCommentForDecl(clang::Decl const*, clang::Preprocessor const*) const llvm-project/clang/lib/AST/ASTContext.cpp:714:35
    #13 in clang_Cursor_getParsedComment llvm-project/clang/tools/libclang/CXComment.cpp:36:35
    #14 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:756:25

Created using spr 1.3.4

llvmbot · 2025-01-30T04:00:07Z

@llvm/pr-subscribers-clang

Author: Vitaly Buka (vitalybuka)

Changes

We can't guaranty that underlying string is
0-terminated and [String.size()] is even in the
same allocation.

Full diff: https://github.com/llvm/llvm-project/pull/125020.diff

1 Files Affected:

(modified) clang/tools/libclang/CXString.cpp (+1-13)

diff --git a/clang/tools/libclang/CXString.cpp b/clang/tools/libclang/CXString.cpp
index 5e427957a1092b..aaa8f8eeb67a12 100644
--- a/clang/tools/libclang/CXString.cpp
+++ b/clang/tools/libclang/CXString.cpp
@@ -87,19 +87,7 @@ CXString createRef(StringRef String) {
   if (String.empty())
     return createEmpty();
 
-  // If the string is not nul-terminated, we have to make a copy.
-
-  // FIXME: This is doing a one past end read, and should be removed! For memory
-  // we don't manage, the API string can become unterminated at any time outside
-  // our control.
-
-  if (String.data()[String.size()] != 0)
-    return createDup(String);
-
-  CXString Result;
-  Result.data = String.data();
-  Result.private_flags = (unsigned) CXS_Unmanaged;
-  return Result;
+  return createDup(String);
 }
 
 CXString createDup(StringRef String) {

vitalybuka · 2025-01-30T04:09:46Z

It does not look nice for memory usage, but I don't know how critical it's here.
It's it's unacceptable, then __attribute__((no_sanitize("memory"))) is an option.

vitalybuka · 2025-02-03T22:21:43Z

ping

vitalybuka · 2025-02-07T17:51:02Z

ping

efriedma-quic

If CXString createRef(StringRef) is essentially equivalent to CXString createDup(StringRef), should we just get rid of CXString createRef(StringRef)?

vitalybuka · 2025-02-11T00:04:03Z

If CXString createRef(StringRef) is essentially equivalent to CXString createDup(StringRef), should we just get rid of CXString createRef(StringRef)?

I guess this is some C API which we need to maintain? No idea how it's used, and if we can remove it.
I'd prefer to land this as-is, and remove createRef in a follow up PR. In case someone find a good reason too keep both we don't want to revert UB fix. WDYT?

#126683

Created using spr 1.3.4

AaronBallman · 2025-02-11T13:21:05Z

If CXString createRef(StringRef) is essentially equivalent to CXString createDup(StringRef), should we just get rid of CXString createRef(StringRef)?

I guess this is some C API which we need to maintain?

Correct, libclang is a stable API. We would have to deprecate the API and wait a decent amount of time before removing it.

AaronBallman

Please add a release note so that users know about the fix and about the expected performance costs, but otherwise LGTM

efriedma-quic · 2025-02-11T22:10:49Z

The public libclang interface needs to be stable, yes... but in this context, that's just the interfaces declared in clang/include/clang-c/CXString.h. We can do whatever we want with the internal interface in clang/tools/libclang/CXString.h.

AaronBallman · 2025-02-12T13:26:36Z

The public libclang interface needs to be stable, yes... but in this context, that's just the interfaces declared in clang/include/clang-c/CXString.h. We can do whatever we want with the internal interface in clang/tools/libclang/CXString.h.

Oh, good catch! Agreed. :-)

Created using spr 1.3.4

vitalybuka · 2025-02-13T06:02:18Z

Please add a release note so that users know about the fix and about the expected performance costs, but otherwise LGTM

done

Created using spr 1.3.4

llvm-ci · 2025-02-13T07:16:54Z

LLVM Buildbot has detected a new failure on builder sanitizer-aarch64-linux-bootstrap-asan running on sanitizer-buildbot7 while building clang at step 2 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/24/builds/5252

Here is the relevant piece of the build log for the reference

Step 2 (annotate) failure: 'python ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py' (failure)
...
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using lld-link: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/lld-link
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld64.lld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/ld64.lld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using wasm-ld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/wasm-ld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld.lld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/ld.lld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using lld-link: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/lld-link
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld64.lld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/ld64.lld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using wasm-ld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/wasm-ld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 86445 tests, 72 workers --
Testing:  0.. 10
FAIL: Clang :: Index/USR/structural-value-tpl-arg.cpp (12308 of 86445)
******************** TEST 'Clang :: Index/USR/structural-value-tpl-arg.cpp' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
RUN: at line 1: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/c-index-test -test-load-source-usrs local -std=c++20 -- /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp | /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp
+ /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp
+ /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/c-index-test -test-load-source-usrs local -std=c++20 -- /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp

--

********************
Testing:  0.. 10
FAIL: Clang :: Index/annotate-attribute.cpp (12311 of 86445)
******************** TEST 'Clang :: Index/annotate-attribute.cpp' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 1: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/c-index-test -test-load-source all /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp | /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp
+ /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp
+ /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/c-index-test -test-load-source all /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp
/home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp:25:11: error: CHECK: expected string not found in input
// CHECK: ClassDecl=Test:3:7 (Definition) Extent=[3:1 - 17:2]
          ^
<stdin>:1:1: note: scanning from here
// CHECK: <invalid loc>:2:9: macro definition=__llvm__
^
<stdin>:1:30: note: possible intended match here
// CHECK: <invalid loc>:2:9: macro definition=__llvm__
                             ^

Input file: <stdin>
Check file: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp

-dump-input=help explains the following input dump.
Step 11 (stage2/asan check) failure: stage2/asan check (failure)
...
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using lld-link: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/lld-link
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld64.lld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/ld64.lld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using wasm-ld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/wasm-ld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld.lld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/ld.lld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using lld-link: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/lld-link
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld64.lld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/ld64.lld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using wasm-ld: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/wasm-ld
llvm-lit: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 86445 tests, 72 workers --
Testing:  0.. 10
FAIL: Clang :: Index/USR/structural-value-tpl-arg.cpp (12308 of 86445)
******************** TEST 'Clang :: Index/USR/structural-value-tpl-arg.cpp' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
RUN: at line 1: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/c-index-test -test-load-source-usrs local -std=c++20 -- /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp | /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp
+ /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp
+ /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/c-index-test -test-load-source-usrs local -std=c++20 -- /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/USR/structural-value-tpl-arg.cpp

--

********************
Testing:  0.. 10
FAIL: Clang :: Index/annotate-attribute.cpp (12311 of 86445)
******************** TEST 'Clang :: Index/annotate-attribute.cpp' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 1: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/c-index-test -test-load-source all /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp | /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp
+ /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp
+ /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/c-index-test -test-load-source all /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp
/home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp:25:11: error: CHECK: expected string not found in input
// CHECK: ClassDecl=Test:3:7 (Definition) Extent=[3:1 - 17:2]
          ^
<stdin>:1:1: note: scanning from here
// CHECK: <invalid loc>:2:9: macro definition=__llvm__
^
<stdin>:1:30: note: possible intended match here
// CHECK: <invalid loc>:2:9: macro definition=__llvm__
                             ^

Input file: <stdin>
Check file: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/test/Index/annotate-attribute.cpp

-dump-input=help explains the following input dump.

We can't guaranty that underlying string is 0-terminated and [String.size()] is even in the same allocation. https://lab.llvm.org/buildbot/#/builders/94/builds/4152/steps/17/logs/stdio ``` ==c-index-test==1846256==WARNING: MemorySanitizer: use-of-uninitialized-value #0 in clang::cxstring::createRef(llvm::StringRef) llvm-project/clang/tools/libclang/CXString.cpp:96:36 llvm#1 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:521:39 llvm#2 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#3 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#4 in DumpCXComment llvm-project/clang/tools/c-index-test/c-index-test.c:685:3 llvm#5 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:768:7 Memory was marked as uninitialized #0 in __msan_allocated_memory llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:1023:5 llvm#1 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:172:7 llvm#2 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:216:12 llvm#3 in Allocate llvm-project/llvm/include/llvm/Support/AllocatorBase.h:53:43 llvm#4 in Allocate<char> llvm-project/llvm/include/llvm/Support/AllocatorBase.h:76:29 llvm#5 in convertCodePointToUTF8 llvm-project/clang/lib/AST/CommentLexer.cpp:42:30 llvm#6 in clang::comments::Lexer::resolveHTMLDecimalCharacterReference(llvm::StringRef) const llvm-project/clang/lib/AST/CommentLexer.cpp:76:10 llvm#7 in clang::comments::Lexer::lexHTMLCharacterReference(clang::comments::Token&) llvm-project/clang/lib/AST/CommentLexer.cpp:615:16 llvm#8 in consumeToken llvm-project/clang/include/clang/AST/CommentParser.h:62:9 llvm#9 in clang::comments::Parser::parseParagraphOrBlockCommand() llvm-project/clang/lib/AST/CommentParser.cpp llvm#10 in clang::comments::Parser::parseFullComment() llvm-project/clang/lib/AST/CommentParser.cpp:925:22 llvm#11 in clang::RawComment::parse(clang::ASTContext const&, clang::Preprocessor const*, clang::Decl const*) const llvm-project/clang/lib/AST/RawCommentList.cpp:221:12 llvm#12 in clang::ASTContext::getCommentForDecl(clang::Decl const*, clang::Preprocessor const*) const llvm-project/clang/lib/AST/ASTContext.cpp:714:35 llvm#13 in clang_Cursor_getParsedComment llvm-project/clang/tools/libclang/CXComment.cpp:36:35 llvm#14 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:756:25 ```

This reverts commit e76739e.

Reverts #125020 https://lab.llvm.org/buildbot/#/builders/24/builds/5252/steps/12/logs/stdio ``` ==c-index-test==2512295==ERROR: AddressSanitizer: heap-use-after-free on address 0xe19338c27992 at pc 0xc66be4784830 bp 0xe0e33660df00 sp 0xe0e33660d6e8 READ of size 23 at 0xe19338c27992 thread T1 #0 0xc66be478482c in printf_common(void*, char const*, std::__va_list) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_format.inc:563:9 #1 0xc66be478643c in vprintf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1699:1 #2 0xc66be478643c in printf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1757:1 #3 0xc66be4839384 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1359:5 #4 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 #5 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp 0xe19338c27992 is located 82 bytes inside of 105-byte region [0xe19338c27940,0xe19338c279a9) freed by thread T1 here: #0 0xc66be480040c in free /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:51:3 #1 0xc66be4839728 in GetCursorSource c-index-test.c #2 0xc66be4839368 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1360:12 #3 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 #4 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp previously allocated by thread T1 here: #0 0xc66be4800680 in malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3 #1 0xe4e3456379b0 in safe_malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/include/llvm/Support/MemAlloc.h:26:18 #2 0xe4e3456379b0 in createDup /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:95:40 #3 0xe4e3456379b0 in clang::cxstring::createRef(llvm::StringRef) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:90:10 ```

…27076) Reverts llvm/llvm-project#125020 https://lab.llvm.org/buildbot/#/builders/24/builds/5252/steps/12/logs/stdio ``` ==c-index-test==2512295==ERROR: AddressSanitizer: heap-use-after-free on address 0xe19338c27992 at pc 0xc66be4784830 bp 0xe0e33660df00 sp 0xe0e33660d6e8 READ of size 23 at 0xe19338c27992 thread T1 #0 0xc66be478482c in printf_common(void*, char const*, std::__va_list) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_format.inc:563:9 #1 0xc66be478643c in vprintf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1699:1 #2 0xc66be478643c in printf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1757:1 #3 0xc66be4839384 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1359:5 #4 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 #5 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp 0xe19338c27992 is located 82 bytes inside of 105-byte region [0xe19338c27940,0xe19338c279a9) freed by thread T1 here: #0 0xc66be480040c in free /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:51:3 #1 0xc66be4839728 in GetCursorSource c-index-test.c #2 0xc66be4839368 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1360:12 #3 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 #4 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp previously allocated by thread T1 here: #0 0xc66be4800680 in malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3 #1 0xe4e3456379b0 in safe_malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/include/llvm/Support/MemAlloc.h:26:18 #2 0xe4e3456379b0 in createDup /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:95:40 #3 0xe4e3456379b0 in clang::cxstring::createRef(llvm::StringRef) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:90:10 ```

We can't guaranty that underlying string is 0-terminated and [String.size()] is even in the same allocation. https://lab.llvm.org/buildbot/#/builders/94/builds/4152/steps/17/logs/stdio ``` ==c-index-test==1846256==WARNING: MemorySanitizer: use-of-uninitialized-value #0 in clang::cxstring::createRef(llvm::StringRef) llvm-project/clang/tools/libclang/CXString.cpp:96:36 #1 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:521:39 llvm#2 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#3 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#4 in DumpCXComment llvm-project/clang/tools/c-index-test/c-index-test.c:685:3 llvm#5 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:768:7 Memory was marked as uninitialized #0 in __msan_allocated_memory llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:1023:5 #1 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:172:7 llvm#2 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:216:12 llvm#3 in Allocate llvm-project/llvm/include/llvm/Support/AllocatorBase.h:53:43 llvm#4 in Allocate<char> llvm-project/llvm/include/llvm/Support/AllocatorBase.h:76:29 llvm#5 in convertCodePointToUTF8 llvm-project/clang/lib/AST/CommentLexer.cpp:42:30 llvm#6 in clang::comments::Lexer::resolveHTMLDecimalCharacterReference(llvm::StringRef) const llvm-project/clang/lib/AST/CommentLexer.cpp:76:10 llvm#7 in clang::comments::Lexer::lexHTMLCharacterReference(clang::comments::Token&) llvm-project/clang/lib/AST/CommentLexer.cpp:615:16 llvm#8 in consumeToken llvm-project/clang/include/clang/AST/CommentParser.h:62:9 llvm#9 in clang::comments::Parser::parseParagraphOrBlockCommand() llvm-project/clang/lib/AST/CommentParser.cpp llvm#10 in clang::comments::Parser::parseFullComment() llvm-project/clang/lib/AST/CommentParser.cpp:925:22 llvm#11 in clang::RawComment::parse(clang::ASTContext const&, clang::Preprocessor const*, clang::Decl const*) const llvm-project/clang/lib/AST/RawCommentList.cpp:221:12 llvm#12 in clang::ASTContext::getCommentForDecl(clang::Decl const*, clang::Preprocessor const*) const llvm-project/clang/lib/AST/ASTContext.cpp:714:35 llvm#13 in clang_Cursor_getParsedComment llvm-project/clang/tools/libclang/CXComment.cpp:36:35 llvm#14 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:756:25 ```

Reverts llvm#125020 https://lab.llvm.org/buildbot/#/builders/24/builds/5252/steps/12/logs/stdio ``` ==c-index-test==2512295==ERROR: AddressSanitizer: heap-use-after-free on address 0xe19338c27992 at pc 0xc66be4784830 bp 0xe0e33660df00 sp 0xe0e33660d6e8 READ of size 23 at 0xe19338c27992 thread T1 #0 0xc66be478482c in printf_common(void*, char const*, std::__va_list) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_format.inc:563:9 #1 0xc66be478643c in vprintf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1699:1 llvm#2 0xc66be478643c in printf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1757:1 llvm#3 0xc66be4839384 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1359:5 llvm#4 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 llvm#5 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp 0xe19338c27992 is located 82 bytes inside of 105-byte region [0xe19338c27940,0xe19338c279a9) freed by thread T1 here: #0 0xc66be480040c in free /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:51:3 #1 0xc66be4839728 in GetCursorSource c-index-test.c llvm#2 0xc66be4839368 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1360:12 llvm#3 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 llvm#4 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp previously allocated by thread T1 here: #0 0xc66be4800680 in malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3 #1 0xe4e3456379b0 in safe_malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/include/llvm/Support/MemAlloc.h:26:18 llvm#2 0xe4e3456379b0 in createDup /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:95:40 llvm#3 0xe4e3456379b0 in clang::cxstring::createRef(llvm::StringRef) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:90:10 ```

Reverts #127076 to reland #125020. Use-after-free should be fixed here #127063

We can't guaranty that underlying string is 0-terminated and [String.size()] is even in the same allocation. https://lab.llvm.org/buildbot/#/builders/94/builds/4152/steps/17/logs/stdio ``` ==c-index-test==1846256==WARNING: MemorySanitizer: use-of-uninitialized-value #0 in clang::cxstring::createRef(llvm::StringRef) llvm-project/clang/tools/libclang/CXString.cpp:96:36 llvm#1 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:521:39 llvm#2 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#3 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#4 in DumpCXComment llvm-project/clang/tools/c-index-test/c-index-test.c:685:3 llvm#5 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:768:7 Memory was marked as uninitialized #0 in __msan_allocated_memory llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:1023:5 llvm#1 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:172:7 llvm#2 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:216:12 llvm#3 in Allocate llvm-project/llvm/include/llvm/Support/AllocatorBase.h:53:43 llvm#4 in Allocate<char> llvm-project/llvm/include/llvm/Support/AllocatorBase.h:76:29 llvm#5 in convertCodePointToUTF8 llvm-project/clang/lib/AST/CommentLexer.cpp:42:30 llvm#6 in clang::comments::Lexer::resolveHTMLDecimalCharacterReference(llvm::StringRef) const llvm-project/clang/lib/AST/CommentLexer.cpp:76:10 llvm#7 in clang::comments::Lexer::lexHTMLCharacterReference(clang::comments::Token&) llvm-project/clang/lib/AST/CommentLexer.cpp:615:16 llvm#8 in consumeToken llvm-project/clang/include/clang/AST/CommentParser.h:62:9 llvm#9 in clang::comments::Parser::parseParagraphOrBlockCommand() llvm-project/clang/lib/AST/CommentParser.cpp llvm#10 in clang::comments::Parser::parseFullComment() llvm-project/clang/lib/AST/CommentParser.cpp:925:22 llvm#11 in clang::RawComment::parse(clang::ASTContext const&, clang::Preprocessor const*, clang::Decl const*) const llvm-project/clang/lib/AST/RawCommentList.cpp:221:12 llvm#12 in clang::ASTContext::getCommentForDecl(clang::Decl const*, clang::Preprocessor const*) const llvm-project/clang/lib/AST/ASTContext.cpp:714:35 llvm#13 in clang_Cursor_getParsedComment llvm-project/clang/tools/libclang/CXComment.cpp:36:35 llvm#14 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:756:25 ```

Reverts llvm#125020 https://lab.llvm.org/buildbot/#/builders/24/builds/5252/steps/12/logs/stdio ``` ==c-index-test==2512295==ERROR: AddressSanitizer: heap-use-after-free on address 0xe19338c27992 at pc 0xc66be4784830 bp 0xe0e33660df00 sp 0xe0e33660d6e8 READ of size 23 at 0xe19338c27992 thread T1 #0 0xc66be478482c in printf_common(void*, char const*, std::__va_list) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_format.inc:563:9 llvm#1 0xc66be478643c in vprintf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1699:1 llvm#2 0xc66be478643c in printf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1757:1 llvm#3 0xc66be4839384 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1359:5 llvm#4 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 llvm#5 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp 0xe19338c27992 is located 82 bytes inside of 105-byte region [0xe19338c27940,0xe19338c279a9) freed by thread T1 here: #0 0xc66be480040c in free /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:51:3 llvm#1 0xc66be4839728 in GetCursorSource c-index-test.c llvm#2 0xc66be4839368 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1360:12 llvm#3 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 llvm#4 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp previously allocated by thread T1 here: #0 0xc66be4800680 in malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3 llvm#1 0xe4e3456379b0 in safe_malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/include/llvm/Support/MemAlloc.h:26:18 llvm#2 0xe4e3456379b0 in createDup /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:95:40 llvm#3 0xe4e3456379b0 in clang::cxstring::createRef(llvm::StringRef) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:90:10 ```

Reverts llvm#127076 to reland llvm#125020. Use-after-free should be fixed here llvm#127063

We can't guaranty that underlying string is 0-terminated and [String.size()] is even in the same allocation. https://lab.llvm.org/buildbot/#/builders/94/builds/4152/steps/17/logs/stdio ``` ==c-index-test==1846256==WARNING: MemorySanitizer: use-of-uninitialized-value #0 in clang::cxstring::createRef(llvm::StringRef) llvm-project/clang/tools/libclang/CXString.cpp:96:36 llvm#1 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:521:39 llvm#2 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#3 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#4 in DumpCXComment llvm-project/clang/tools/c-index-test/c-index-test.c:685:3 llvm#5 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:768:7 Memory was marked as uninitialized #0 in __msan_allocated_memory llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:1023:5 llvm#1 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:172:7 llvm#2 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:216:12 llvm#3 in Allocate llvm-project/llvm/include/llvm/Support/AllocatorBase.h:53:43 llvm#4 in Allocate<char> llvm-project/llvm/include/llvm/Support/AllocatorBase.h:76:29 llvm#5 in convertCodePointToUTF8 llvm-project/clang/lib/AST/CommentLexer.cpp:42:30 llvm#6 in clang::comments::Lexer::resolveHTMLDecimalCharacterReference(llvm::StringRef) const llvm-project/clang/lib/AST/CommentLexer.cpp:76:10 llvm#7 in clang::comments::Lexer::lexHTMLCharacterReference(clang::comments::Token&) llvm-project/clang/lib/AST/CommentLexer.cpp:615:16 llvm#8 in consumeToken llvm-project/clang/include/clang/AST/CommentParser.h:62:9 llvm#9 in clang::comments::Parser::parseParagraphOrBlockCommand() llvm-project/clang/lib/AST/CommentParser.cpp llvm#10 in clang::comments::Parser::parseFullComment() llvm-project/clang/lib/AST/CommentParser.cpp:925:22 llvm#11 in clang::RawComment::parse(clang::ASTContext const&, clang::Preprocessor const*, clang::Decl const*) const llvm-project/clang/lib/AST/RawCommentList.cpp:221:12 llvm#12 in clang::ASTContext::getCommentForDecl(clang::Decl const*, clang::Preprocessor const*) const llvm-project/clang/lib/AST/ASTContext.cpp:714:35 llvm#13 in clang_Cursor_getParsedComment llvm-project/clang/tools/libclang/CXComment.cpp:36:35 llvm#14 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:756:25 ```

Reverts llvm#125020 https://lab.llvm.org/buildbot/#/builders/24/builds/5252/steps/12/logs/stdio ``` ==c-index-test==2512295==ERROR: AddressSanitizer: heap-use-after-free on address 0xe19338c27992 at pc 0xc66be4784830 bp 0xe0e33660df00 sp 0xe0e33660d6e8 READ of size 23 at 0xe19338c27992 thread T1 #0 0xc66be478482c in printf_common(void*, char const*, std::__va_list) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_format.inc:563:9 llvm#1 0xc66be478643c in vprintf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1699:1 llvm#2 0xc66be478643c in printf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1757:1 llvm#3 0xc66be4839384 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1359:5 llvm#4 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 llvm#5 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp 0xe19338c27992 is located 82 bytes inside of 105-byte region [0xe19338c27940,0xe19338c279a9) freed by thread T1 here: #0 0xc66be480040c in free /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:51:3 llvm#1 0xc66be4839728 in GetCursorSource c-index-test.c llvm#2 0xc66be4839368 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1360:12 llvm#3 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 llvm#4 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp previously allocated by thread T1 here: #0 0xc66be4800680 in malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3 llvm#1 0xe4e3456379b0 in safe_malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/include/llvm/Support/MemAlloc.h:26:18 llvm#2 0xe4e3456379b0 in createDup /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:95:40 llvm#3 0xe4e3456379b0 in clang::cxstring::createRef(llvm::StringRef) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:90:10 ```

Reverts llvm#127076 to reland llvm#125020. Use-after-free should be fixed here llvm#127063

We can't guaranty that underlying string is 0-terminated and [String.size()] is even in the same allocation. https://lab.llvm.org/buildbot/#/builders/94/builds/4152/steps/17/logs/stdio ``` ==c-index-test==1846256==WARNING: MemorySanitizer: use-of-uninitialized-value #0 in clang::cxstring::createRef(llvm::StringRef) llvm-project/clang/tools/libclang/CXString.cpp:96:36 llvm#1 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:521:39 llvm#2 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#3 in DumpCXCommentInternal llvm-project/clang/tools/c-index-test/c-index-test.c:674:7 llvm#4 in DumpCXComment llvm-project/clang/tools/c-index-test/c-index-test.c:685:3 llvm#5 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:768:7 Memory was marked as uninitialized #0 in __msan_allocated_memory llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:1023:5 llvm#1 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:172:7 llvm#2 in Allocate llvm-project/llvm/include/llvm/Support/Allocator.h:216:12 llvm#3 in Allocate llvm-project/llvm/include/llvm/Support/AllocatorBase.h:53:43 llvm#4 in Allocate<char> llvm-project/llvm/include/llvm/Support/AllocatorBase.h:76:29 llvm#5 in convertCodePointToUTF8 llvm-project/clang/lib/AST/CommentLexer.cpp:42:30 llvm#6 in clang::comments::Lexer::resolveHTMLDecimalCharacterReference(llvm::StringRef) const llvm-project/clang/lib/AST/CommentLexer.cpp:76:10 llvm#7 in clang::comments::Lexer::lexHTMLCharacterReference(clang::comments::Token&) llvm-project/clang/lib/AST/CommentLexer.cpp:615:16 llvm#8 in consumeToken llvm-project/clang/include/clang/AST/CommentParser.h:62:9 llvm#9 in clang::comments::Parser::parseParagraphOrBlockCommand() llvm-project/clang/lib/AST/CommentParser.cpp llvm#10 in clang::comments::Parser::parseFullComment() llvm-project/clang/lib/AST/CommentParser.cpp:925:22 llvm#11 in clang::RawComment::parse(clang::ASTContext const&, clang::Preprocessor const*, clang::Decl const*) const llvm-project/clang/lib/AST/RawCommentList.cpp:221:12 llvm#12 in clang::ASTContext::getCommentForDecl(clang::Decl const*, clang::Preprocessor const*) const llvm-project/clang/lib/AST/ASTContext.cpp:714:35 llvm#13 in clang_Cursor_getParsedComment llvm-project/clang/tools/libclang/CXComment.cpp:36:35 llvm#14 in PrintCursorComments llvm-project/clang/tools/c-index-test/c-index-test.c:756:25 ```

Reverts llvm#125020 https://lab.llvm.org/buildbot/#/builders/24/builds/5252/steps/12/logs/stdio ``` ==c-index-test==2512295==ERROR: AddressSanitizer: heap-use-after-free on address 0xe19338c27992 at pc 0xc66be4784830 bp 0xe0e33660df00 sp 0xe0e33660d6e8 READ of size 23 at 0xe19338c27992 thread T1 #0 0xc66be478482c in printf_common(void*, char const*, std::__va_list) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_format.inc:563:9 llvm#1 0xc66be478643c in vprintf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1699:1 llvm#2 0xc66be478643c in printf /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1757:1 llvm#3 0xc66be4839384 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1359:5 llvm#4 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 llvm#5 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp 0xe19338c27992 is located 82 bytes inside of 105-byte region [0xe19338c27940,0xe19338c279a9) freed by thread T1 here: #0 0xc66be480040c in free /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:51:3 llvm#1 0xc66be4839728 in GetCursorSource c-index-test.c llvm#2 0xc66be4839368 in FilteredPrintingVisitor /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/c-index-test/c-index-test.c:1360:12 llvm#3 0xe4e3454f12e8 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CIndex.cpp:227:11 llvm#4 0xe4e3454f48a8 in bool clang::cxcursor::CursorVisitor::visitPreprocessedEntities<clang::PreprocessingRecord::iterator>(clang::PreprocessingRecord::iterator, clang::PreprocessingRecord::iterator, clang::PreprocessingRecord&, clang::FileID) CIndex.cpp previously allocated by thread T1 here: #0 0xc66be4800680 in malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3 llvm#1 0xe4e3456379b0 in safe_malloc /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/include/llvm/Support/MemAlloc.h:26:18 llvm#2 0xe4e3456379b0 in createDup /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:95:40 llvm#3 0xe4e3456379b0 in clang::cxstring::createRef(llvm::StringRef) /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/clang/tools/libclang/CXString.cpp:90:10 ```

Reverts llvm#127076 to reland llvm#125020. Use-after-free should be fixed here llvm#127063

[𝘀𝗽𝗿] initial version

10af910

Created using spr 1.3.4

llvmbot added clang Clang issues not falling into any other category clang:as-a-library libclang and C++ API labels Jan 30, 2025

vitalybuka requested review from egorzhdan, gribozavr and Arnaud-de-Grandmaison-ARM January 30, 2025 04:01

vitalybuka requested a review from AaronBallman January 30, 2025 04:13

vitalybuka changed the title ~~[libclang] Always Dup in createRef~~ [libclang] Always Dup in createRef(StringRef) Jan 30, 2025

vitalybuka requested review from alexey-bataev and efriedma-quic February 8, 2025 01:55

efriedma-quic reviewed Feb 10, 2025

View reviewed changes

vitalybuka requested a review from efriedma-quic February 11, 2025 00:04

rebase

fe55503

Created using spr 1.3.4

AaronBallman approved these changes Feb 11, 2025

View reviewed changes

relnote

3ece7bc

Created using spr 1.3.4

vitalybuka added 2 commits February 12, 2025 22:03

quotes

278a41b

Created using spr 1.3.4

the

5fd15be

Created using spr 1.3.4

vitalybuka merged commit e76739e into main Feb 13, 2025
7 of 9 checks passed

vitalybuka deleted the users/vitalybuka/spr/libclang-always-dup-in-createref branch February 13, 2025 06:05

vitalybuka restored the users/vitalybuka/spr/libclang-always-dup-in-createref branch February 13, 2025 15:39

vitalybuka deleted the users/vitalybuka/spr/libclang-always-dup-in-createref branch February 13, 2025 15:39

vitalybuka added a commit that referenced this pull request Feb 13, 2025

Revert "[libclang] Always Dup in createRef(StringRef) (#125020)"

3548823

This reverts commit e76739e.

vitalybuka mentioned this pull request Feb 13, 2025

Revert "[libclang] Always Dup in createRef(StringRef)" #127076

Merged

vitalybuka mentioned this pull request Feb 13, 2025

Reland "[libclang] Always Dup in createRef(StringRef)" #127078

Merged

vitalybuka added a commit that referenced this pull request Feb 20, 2025

Reland "[libclang] Always Dup in createRef(StringRef)" (#127078)

a16fa3a

Reverts #127076 to reland #125020. Use-after-free should be fixed here #127063

vitalybuka mentioned this pull request Feb 25, 2025

[libclang] Replace createRef with createDup #126683

Open

joaosaffran pushed a commit to joaosaffran/llvm-project that referenced this pull request Feb 27, 2025

Reland "[libclang] Always Dup in createRef(StringRef)" (llvm#127078)

8355861

Reverts llvm#127076 to reland llvm#125020. Use-after-free should be fixed here llvm#127063

YutongZhuu pushed a commit to YutongZhuu/llvm-project that referenced this pull request Mar 8, 2025

Reland "[libclang] Always Dup in createRef(StringRef)" (llvm#127078)

f6e00d4

Reverts llvm#127076 to reland llvm#125020. Use-after-free should be fixed here llvm#127063

YutongZhuu pushed a commit to YutongZhuu/llvm-project that referenced this pull request Mar 8, 2025

Reland "[libclang] Always Dup in createRef(StringRef)" (llvm#127078)

8fd3ba8

Reverts llvm#127076 to reland llvm#125020. Use-after-free should be fixed here llvm#127063

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[libclang] Always Dup in createRef(StringRef) #125020

[libclang] Always Dup in createRef(StringRef) #125020

vitalybuka commented Jan 30, 2025 •

edited

Loading

llvmbot commented Jan 30, 2025

vitalybuka commented Jan 30, 2025

vitalybuka commented Feb 3, 2025

vitalybuka commented Feb 7, 2025

efriedma-quic left a comment

vitalybuka commented Feb 11, 2025 •

edited

Loading

AaronBallman commented Feb 11, 2025

AaronBallman left a comment

efriedma-quic commented Feb 11, 2025

AaronBallman commented Feb 12, 2025

vitalybuka commented Feb 13, 2025

llvm-ci commented Feb 13, 2025

[libclang] Always Dup in createRef(StringRef) #125020

[libclang] Always Dup in createRef(StringRef) #125020

Conversation

vitalybuka commented Jan 30, 2025 • edited Loading

llvmbot commented Jan 30, 2025

vitalybuka commented Jan 30, 2025

vitalybuka commented Feb 3, 2025

vitalybuka commented Feb 7, 2025

efriedma-quic left a comment

Choose a reason for hiding this comment

vitalybuka commented Feb 11, 2025 • edited Loading

AaronBallman commented Feb 11, 2025

AaronBallman left a comment

Choose a reason for hiding this comment

efriedma-quic commented Feb 11, 2025

AaronBallman commented Feb 12, 2025

vitalybuka commented Feb 13, 2025

llvm-ci commented Feb 13, 2025

vitalybuka commented Jan 30, 2025 •

edited

Loading

vitalybuka commented Feb 11, 2025 •

edited

Loading