Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native crash running Java.Interop.Export-Tests.dll tests #514

Open
jpobst opened this issue Nov 4, 2019 · 1 comment
Open

Native crash running Java.Interop.Export-Tests.dll tests #514

jpobst opened this issue Nov 4, 2019 · 1 comment
Labels
bug Component does not function as intended java-interop Runtime bridge between .NET and Java

Comments

@jpobst
Copy link
Contributor

jpobst commented Nov 4, 2019

When CI is running bin/TestDebug/Java.Interop.Export-Tests.dll on Mac, there's about a 25% chance it fails with this native crash:

  * Assertion: should not be reached at class-accessors.c:81
  
  
  =================================================================
  	Native Crash Reporting
  =================================================================
  Got a SIGABRT while executing native code. This usually indicates
  a fatal error in the mono runtime or one of the native libraries 
  used by your application.
  =================================================================
  
  =================================================================
  	Native stacktrace:
  =================================================================
  	0x102d2091b - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_dump_native_crash_info
  	0x102cb97b5 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_handle_native_crash
  	0x102d1fe71 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : sigabrt_signal_handler
  	0x7fff79156b5d - /usr/lib/system/libsystem_platform.dylib : _sigtramp
  	0x10a126f87 - Unknown
  	0x7fff790106a6 - /usr/lib/system/libsystem_c.dylib : abort
  	0x102f28837 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : monoeg_assert_abort
  	0x102f09a9f - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_log_write_logfile
  	0x102f28bbe - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : monoeg_g_logv_nofree
  	0x102f28d3f - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : monoeg_assertion_message
  	0x102db6ee5 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_get_flags
  	0x102dac83b - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_setup_fields
  	0x102dacdc1 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_init_internal
  	0x102da7e7f - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_is_subclass_of_internal
  	0x102e2e4f3 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_is_subclass_of
  	0x106bf1348 - /Users/runner/runners/2.160.0/work/1/s/bin/TestDebug/libjava-interop.dylib : _ZL19get_gc_bridge_indexP19JavaInteropGCBridgeP10_MonoClass
  	0x106bf0a4c - /Users/runner/runners/2.160.0/work/1/s/bin/TestDebug/libjava-interop.dylib : _ZL20gc_bridge_class_kindP10_MonoClass
  SIGQUIT: [libjvm.dylib+0x49bf1d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
    Start time: 2019-11-04 19:57:01Z
      End time: 2019-11-04 19:57:02Z
      Duration: 1.803 seconds

Some examples:

@jpobst jpobst changed the title Native crash running tests Native crash running Java.Interop.Export-Tests.dll tests Nov 4, 2019
@jonpryor
Copy link
Member

jonpryor commented Nov 4, 2019

Related: mono/mono#14190

The crash appears to happen because we're trying to do a GC during process shutdown.

Possible WorkAround: Don't Do That™. If we could tell if mono were shutting down the process, we could skip the entire GC infrastructure -- not GC'ing -- during process shutdown, thus avoiding the crash.

The question is, How do we tell if the process is shutting down. Looks like the only public mechanism is to use the "profiler", with the MonoProfilerRuntimeShutdownCallback event.

Looks like there should be a mono_profiler_set_runtime_shutdown_callback() function...?

@jpobst jpobst added bug Component does not function as intended java-interop Runtime bridge between .NET and Java labels Apr 6, 2020
jonpryor pushed a commit that referenced this issue Apr 11, 2023
Context: #514

There is a known native crash that occasionally occurs when running
unit tests on CI; see Issue #514:

	* Assertion: should not be reached at class-accessors.c:81
	 
	 
	 =================================================================
	 	Native Crash Reporting
	 =================================================================
	 Got a SIGABRT while executing native code. This usually indicates
	 a fatal error in the mono runtime or one of the native libraries 
	 used by your application.
	 =================================================================
	 
	 =================================================================
	 	Native stacktrace:
	 =================================================================
	 	0x102d2091b - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_dump_native_crash_info
	 	0x102cb97b5 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_handle_native_crash
	 	0x102d1fe71 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : sigabrt_signal_handler
	 	0x7fff79156b5d - /usr/lib/system/libsystem_platform.dylib : _sigtramp
	 	0x10a126f87 - Unknown
	 	0x7fff790106a6 - /usr/lib/system/libsystem_c.dylib : abort
	 	0x102f28837 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : monoeg_assert_abort
	 	0x102f09a9f - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_log_write_logfile
	 	0x102f28bbe - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : monoeg_g_logv_nofree
	 	0x102f28d3f - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : monoeg_assertion_message
	 	0x102db6ee5 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_get_flags
	 	0x102dac83b - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_setup_fields
	 	0x102dacdc1 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_init_internal
	 	0x102da7e7f - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_is_subclass_of_internal
	 	0x102e2e4f3 - /Library/Frameworks/Mono.framework/Versions/Current/Commands/mono : mono_class_is_subclass_of
	 	0x106bf1348 - /Users/runner/runners/2.160.0/work/1/s/bin/TestDebug/libjava-interop.dylib : _ZL19get_gc_bridge_indexP19JavaInteropGCBridgeP10_MonoClass
	 	0x106bf0a4c - /Users/runner/runners/2.160.0/work/1/s/bin/TestDebug/libjava-interop.dylib : _ZL20gc_bridge_class_kindP10_MonoClass
	 SIGQUIT: [libjvm.dylib+0x49bf1d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
	   Start time: 2019-11-04 19:57:01Z
	     End time: 2019-11-04 19:57:02Z
	     Duration: 1.803 seconds

This crash doesn't happen "in the wild" so it is unlikely to get
fixed anytime soon.

As a workaround for our CI, we will use AzDO's 
[`retryCountOnTaskFailure` parameter][0] to automatically re-run a
native test suite once if it fails.  This should reduce the need for
us to re-run failed pipelines for flaky tests.

Note that AzDO's implementation isn't perfect; the task will succeed
if there are no failures on retry, but the original failures will
still be listed in the log.  🤷‍♂️ 

[0]: https://learn.microsoft.com/en-us/azure/devops/release-notes/2021/pipelines/sprint-195-update#automatic-retries-for-a-task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Component does not function as intended java-interop Runtime bridge between .NET and Java
Projects
None yet
Development

No branches or pull requests

2 participants