Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java module errors: no jnilept in java.library.path / cannot access class TessBaseAPI #814

Closed
breandan opened this issue Nov 20, 2019 · 31 comments
Assignees
Labels

Comments

@breandan
Copy link

When I declare the following module-info.java on JDK 13:

module example {
    //...
    requires org.bytedeco.javacv;
    requires org.bytedeco.tesseract;
    //...
}

I receive the following error: no jnilept in java.library.path: [/Developer/NVIDIA/CUDA-9.1/lib, /Users/breandan/Library/Java/Extensions, /Library/Java/Extensions, /Network/Library/Java/Extensions, /System/Library/Java/Extensions, /usr/lib/java, .]

When I modify the module-info.java to use the .platform dependency:

module example {
    //...
    requires org.bytedeco.javacv;
    requires org.bytedeco.tesseract.platform;
    //...
}

The following error occurs: cannot access class org.bytedeco.tesseract.TessBaseAPI (in module org.bytedeco.tesseract) because module example does not read module org.bytedeco.tesseract.

I have tried a bunch of variations of transitive and requiring both dependencies, to no avail. Do you have any suggestions? Thank you!

@saudet
Copy link
Member

saudet commented Nov 20, 2019

I'm pretty sure the sample project here works:
https://github.com/bytedeco/sample-projects/tree/master/opencv-stitching-jlink
What does that one give for you?

@saudet
Copy link
Member

saudet commented Nov 20, 2019

It simply looks like Leptonica is missing from the class/module path though.
Have you tried to add that one explicitly as well?

@breandan
Copy link
Author

It simply looks like Leptonica is missing from the class/module path though. Have you tried to add that one explicitly as well?

This worked! I added org.bytedeco.leptonica.platform and the linker was finally satisfied. Not sure why I didn't think to do that in the first place, I assumed that using a transitive dependency would have sufficed, but apparently modules do not work as I expected.

By the way, I am sitting just a few floors below your old lab in McConnell Engineering. From one McGill grad student to another, thanks for publishing javacpp and all the presets. I've been following your work on Bytedeco for some time, keep up the great work!

@saudet
Copy link
Member

saudet commented Nov 21, 2019

I see, thanks for testing! Looking more closely as this, I think we could add a requires transitive org.bytedeco.leptonica.platform here:
https://github.com/bytedeco/javacpp-presets/blob/master/tesseract/platform/pom.xml#L161
Could you give this a try and send a pull request if that works as expected? We can easily build the "-platform" artifacts without recompiling anything. It just downloads everything, which I haven't found a way to reduce yet in the case of the module path, but we can do this when using the class path:
https://github.com/bytedeco/javacpp-presets/wiki/Reducing-the-Number-of-Dependencies

You're a student working as part of MILA? I'm glad to hear that my work is useful there! Let me know if there's anything I can do to help you guys more. Thanks

/cc @HGuillemet

@saudet saudet reopened this Nov 21, 2019
@saudet saudet added the bug label Nov 21, 2019
@breandan
Copy link
Author

breandan commented Nov 21, 2019

Okay, so I forked and cloned the parent repo as suggested, added the line requires transitive org.bytedeco.leptonica.platform to the javacpp-presets/tesseract/platform/pom.xml file, then ran mvn install from the directory, imported the new version (4.1.0-1.5.3-SNAPSHOT as of 4d374b5), and tried rebuilding with the following two requirements in the module declaration:

...
    requires org.bytedeco.javacv;
    requires org.bytedeco.tesseract.platform;
...

but encountered the same error as before (cannot access class ...TessBaseAPI). Only after adding the platform binary was the error finally resolved. Actually, the specific platform binary is required, since the org.bytedeco.leptonica.platform dependency introduced another mysterious error:

java.lang.LayerInstantiationException: Package lib.x86 in both module org.bytedeco.leptonica.android.x86 and module org.bytedeco.tesseract.android.x86

I slightly regret upgrading to Java 9+, but wanted to take advantage of the new jlink/jpackage functionality. Anyhow, I appreciate your earlier support, which led to the correct solution.

Re: Mila/JavaCPP. Speaking for myself, but it's definitely been helpful. Zero configuration, very straightforward to setup and use. We've been discussing adding GPU support for our AD library, in which case we might use the CUDA presets. Right now, just using it for some side projects, but it's probably the smoothest FFI I've ever had to use. Thanks again for your time and energy on this project.

@saudet
Copy link
Member

saudet commented Nov 21, 2019

I see, so org.bytedeco.leptonica.platform and org.bytedeco.tesseract.platform conflict with each other? The lib package is used by Android to store native libraries, but since it doesn't support JPMS, yet anyway, maybe we can drop its artifacts from the list of modules... @HGuillemet What do you think?

JPMS is very much clunky at this stage, yes. We can still use the class path with Java 9+ though. It's also possible to use jlink without modules, by creating a single module from an uber JAR and feeding that to jlink. Until more libraries start supporting JPMS fully, that's probably the way to go, and given that the Java community at large hasn't accepted JPMS yet, it might stay that way for a while:
https://www.theregister.co.uk/2017/05/12/oracle_loses_jpms_vote/

Your project looks very interesting! I'm sure @treo will start looking at that right away :) Thanks for sharing and keep us informed of your progress.

@HGuillemet
Copy link
Collaborator

The tesseract.platform module has dependencies towards all native modules. The tesseract module does not. So using the tesseract module is the way to go when you build for specific platform(s) and want to avoid all foreign native modules. You should be able to simple do:

requires org.bytedeco.tesseract.macosx.x86-64;

instead of:

requires org.bytedeco.tesseract;

(no need for both, the native module has a transitive dependency towards the java one).
If you prefer a build that works anywhere, use the other option instead:

requires org.bytedeco.tesseract.platform;

That said, neither work as is because the tesseract native modules should "require" the leptonica native modules since the tesseract library is linked towards the leptonica native library, and it doesn't. I think adding the requires to each tesseract native module would be better that adding it to the platform module only, since it would allow both options above to work as expected.
We can do this by overriding the moditect section of the parent pom in the tesseract pom, but that's a bunch of lines to copy to just add 1 line:

      <plugin>
        <groupId>org.moditect</groupId>
        <artifactId>moditect-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>add-module-infos</id>
            <phase>package</phase>
            <goals>
              <goal>add-module-info</goal>
            </goals>
            <configuration>
              <modules>
                <module>
                  <file>${project.build.directory}/${project.artifactId}.jar</file>
                  <moduleInfoFile>${project.basedir}/src/main/java9/module-info.java</moduleInfoFile>
                </module>
                <module>
                  <file>${project.build.directory}/${project.artifactId}-${javacpp.platform}${javacpp.platform.extension}.jar</file>
                  <moduleInfoSource>
                    open module org.bytedeco.${javacpp.packageName}.${javacpp.platform.module} {
                      requires transitive org.bytedeco.${javacpp.packageName};
                      requires org.bytedeco.leptonica.${javacpp.platform.module};
                    }
                  </moduleInfoSource>
                </module>
              </modules>
            </configuration>
          </execution>
        </executions>
      </plugin>

Since this situation should arises in many other presets, we can simplify and instead define a new maven property. Something like:

  <properties>
    <project.additionalRequires>requires org.bytedeco.leptonica.${javacpp.platform.module};</project.additionalRequires>
  </properties>

and use this property in the moditect section of parent pom.
Samuel, any better idea ?

@HGuillemet
Copy link
Collaborator

Concerning the android specific problem: in modules for android, native libraries are located in /lib/<abi>, because Android expects them to be here. For the x86 abi, this gives /lib/x86.

lib.x86 is a valid Java package, so it's added to the list of packages of this module. Thus the split-package error when using more than 1 native modules.
This doesn't happen with non-android modules because libraries are located in a path always containing a -, and with android modules with an abi with -.
See this discussion on jigsaw-dev

Solutions I see:

  1. Explicitly set the ModulePackages attribute in the ModuleInfo to exclude lib.abi. Not sure how to do this cleanly.
  2. Keep the same path for libraries in the native jar for android than for other plaforms, and let JavaCPP install them in /lib/abi when it explodes the jar. Samuel, is this possible ?
  3. Say JPMS is unsupported for JavaCPP on Android.

@saudet
Copy link
Member

saudet commented Nov 22, 2019 via email

@HGuillemet
Copy link
Collaborator

PR for the native dependency

I don't really understand the problem with Android. If JPMS is not implemented on Android, how the split package error can show up ?

Are you sure there isn't a simple workaround to this error, similar to the use of org.bytedeco.javacpp.cachedir.nosubdir, that could extract the libraries to /lib/abi without sub directories ?

@saudet
Copy link
Member

saudet commented Nov 24, 2019

JPMS is probably going to complain about having non-modular JAR files in its path, no?

Android actually "installs" libraries in the lib subdirectory from an AAR artifact,
see bytedeco/javacv#1117. So we don't technically need JAR files, but...

@HGuillemet
Copy link
Collaborator

If the android jars do not contain a module-info, that could work. The split package error with lib.x86 because in unnamed modules /lib/x86 directory won't be considered as a package if it doesn't contain any class. But jlink won't work, since jlink want all the modules to be named (with explicit module-info).

@saudet
Copy link
Member

saudet commented Nov 29, 2019

I can't seem to reproduce that issue with the lib package from the artifacts for Android though.
@breandan Do you have a small example project failing like that somewhere that I can look at?

@breandan
Copy link
Author

breandan commented Nov 29, 2019

Sure, to reproduce the error described above, please run git clone [email protected]:acejump/TraceJump.git && git checkout e0b0cbcb3cb39ca61939e265d7a1a0ea5a75bd19 && ./gradlew run --stacktrace. On JDK 11, I receive the following error:

> Task :run FAILED
Error occurred during initialization of boot layer
java.lang.LayerInstantiationException: Package lib.x86_64 in both module org.bytedeco.tesseract.android.x86_64 and module org.bytedeco.leptonica.android.x86_64

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':run'.
> Process 'command '/Users/breandan/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/192.7142.36/IntelliJ IDEA.app/Contents/jbr/Contents/Home/bin/java'' finished with non-zero exit value 1

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':run'.
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:188)
	at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:263)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:186)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:167)
	at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:109)
	at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
	at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62)
	at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
	at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
	at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
	at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
	at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:374)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:361)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:354)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:340)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
Caused by: org.gradle.process.internal.ExecException: Process 'command '/Users/breandan/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/192.7142.36/IntelliJ IDEA.app/Contents/jbr/Contents/Home/bin/java'' finished with non-zero exit value 1
	at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:417)
	at org.gradle.process.internal.DefaultJavaExecAction.execute(DefaultJavaExecAction.java:40)
	at org.gradle.api.tasks.JavaExec.exec(JavaExec.java:120)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104)
	at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:49)
	at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:42)
	at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:28)
	at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:721)
	at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:688)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$3.run(ExecuteActionsTaskExecuter.java:547)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:402)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:394)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:92)
	at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:532)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:515)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.access$300(ExecuteActionsTaskExecuter.java:109)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.executeWithPreviousOutputFiles(ExecuteActionsTaskExecuter.java:259)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.execute(ExecuteActionsTaskExecuter.java:248)
	at org.gradle.internal.execution.steps.ExecuteStep.lambda$execute$1(ExecuteStep.java:33)
	at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:33)
	at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:26)
	at org.gradle.internal.execution.steps.CleanupOutputsStep.execute(CleanupOutputsStep.java:63)
	at org.gradle.internal.execution.steps.CleanupOutputsStep.execute(CleanupOutputsStep.java:35)
	at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:49)
	at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:34)
	at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:43)
	at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:73)
	at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:54)
	at org.gradle.internal.execution.steps.CatchExceptionStep.execute(CatchExceptionStep.java:34)
	at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:44)
	at org.gradle.internal.execution.steps.SnapshotOutputsStep.execute(SnapshotOutputsStep.java:54)
	at org.gradle.internal.execution.steps.SnapshotOutputsStep.execute(SnapshotOutputsStep.java:38)
	at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:49)
	at org.gradle.internal.execution.steps.CacheStep.executeWithoutCache(CacheStep.java:153)
	at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:67)
	at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:41)
	at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:44)
	at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:33)
	at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:38)
	at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:24)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:92)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$0(SkipUpToDateStep.java:85)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:55)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:39)
	at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:76)
	at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:37)
	at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:36)
	at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:26)
	at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:94)
	at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:49)
	at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:79)
	at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:53)
	at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:74)
	at org.gradle.internal.execution.steps.SkipEmptyWorkStep.lambda$execute$2(SkipEmptyWorkStep.java:78)
	at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:78)
	at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:34)
	at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:39)
	at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:40)
	at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:28)
	at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:33)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:175)
	... 30 more


* Get more help at https://help.gradle.org

BUILD FAILED in 1s

@saudet
Copy link
Member

saudet commented Nov 29, 2019

Thanks! Can you reproduce it outside Gradle with a single jlink command? It doesn't happen with the Maven plugin, so it looks like something wrong with the Gradle plugin, but I can't see why that would make a difference...

@saudet
Copy link
Member

saudet commented Nov 29, 2019

Oh, I see, it's an error that occurs at runtime. Why would it wait for runtime to report build errors? Weird...

How can we fix this? I tried to declare those modules using only module instead of open module, but it looks like JPMS doesn't even allow split packages that are hidden:

  1. Two modules may not contain the same package. This seems eminently sensible, until you consider that it also applies to hidden packages. Since hidden packages are not listed in module-info.class, a tool like Maven must unpack the jar file to discover what hidden packages there are in order to warn of clashes. As a user of the library, such a clash will be completely surprising, as you won’t have any indication of the hidden packages in the Javadoc. This is a more general indication that JPMS does not provide sufficient isolation between modules, for reasons that are far from clear at this point.

Consequently, excluding those JAR files entirely from the platform modules appears to be the only way to solve this, which is fine I guess since Android doesn't support JPMS or jlink anyway.
@HGuillemet What do you think? Can you think of anything else?

@HGuillemet
Copy link
Collaborator

If I understand well, this error only happens at runtime, only when including the android package in the module path, and only when running on a non-android system (as long as android doesn't support JPMS).
So I believe the best ways to fix this are either:

  • like suggested above, to keep the android native library in the same path structure that other platforms (/org/bytedeco/tesseract/android-arm64/) and have them installed in /lib somehow (by javacpp extractor or when using the aar archive). I think there is already the functionality to allow to extract the lib using a directory structure different from the one in the jar.
  • separate the desktop platform from the android platform with maybe 2 classifiers : desktop and android instead of platform. The android platform won't use JPMS and we won't allow jlink. Are there really use cases where we need a build that works on both desktop and android ?

I'd say first option is better because it won't change anything for current users but maybe this is technically too difficult for a reason I don't see.

@saudet
Copy link
Member

saudet commented Nov 29, 2019

Either option would require developer time and increase the cognitive load for end users... For now, just removing the Android modules from the moduleInfoSource for the platform artifacts works around this, so let's do that. We'll need to create AAR files for Android eventually. Let's revisit this once that is done.

@HGuillemet
Copy link
Collaborator

HGuillemet commented Nov 29, 2019

Removing the module-info from the android native jar will prevent from using jlink on a project using a -platform artifact, because jlink will refuse automatic modules.

@saudet
Copy link
Member

saudet commented Nov 29, 2019

We don't need to change those, just the platform artifacts.

@HGuillemet
Copy link
Collaborator

Ok, I misread, sorry.
Yes, that would work.
The jlink-maven-plugin uses the maven dependencies to build the image. So if you keep the android maven dependencies in the platform artifact, the android artifacts will be in the image. But as long as they are not brought in the module graph by a requires, the split package error shouldn't show up.

@saudet
Copy link
Member

saudet commented Dec 19, 2019

@HGuillemet Could you send a pull request for that too? Unless you can think of something better...

@HGuillemet
Copy link
Collaborator

Done. Something better would be IMO as discussed above to keep the same directory structure in native jars for all platforms. And have JavaCPP install the libs in the directory expected by Android, either in Loader.cacheResource or when using a AAR files. But no hurry as long as there is no JPMS on Android.

@saudet
Copy link
Member

saudet commented Dec 23, 2019

Thanks!

We can't put binaries anywhere we want on Android. Well, technically we can do anything we want, but realistically you don't want to go against Google. The only thing that it supports officially are AAR files.

saudet pushed a commit that referenced this issue Dec 24, 2019
@saudet
Copy link
Member

saudet commented Apr 15, 2020

Fix from @HGuillemet has been released with version 1.5.3. Thanks!

And thanks for reporting and for testing @breandan!

@saudet saudet closed this as completed Apr 15, 2020
@saudet
Copy link
Member

saudet commented Jun 27, 2020

BTW, there is now a plugin for Gradle to exclude by platform names:
https://github.com/bytedeco/gradle-javacpp#the-platform-plugin
If you try it out and find anything missing, please let me know!

@breandan
Copy link
Author

Does JavaCPP publish Leptonica presets for macosx-arm64? I tried configuring the new Gradle plugin, but was unable to get past the following error on OSX 12.0.1 with OpenJDK 17.

Exception in thread "Thread-2" java.lang.UnsatisfiedLinkError: no jnilept in java.library.path: /Users/breandan/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2429)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)
        at java.base/java.lang.System.loadLibrary(System.java:1989)
        at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1738)
        at org.bytedeco.javacpp.Loader.load(Loader.java:1345)
        at org.bytedeco.javacpp.Loader.load(Loader.java:1157)
        at org.bytedeco.javacpp.Loader.load(Loader.java:1133)
        at org.bytedeco.leptonica.global.lept.<clinit>(lept.java:14)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:467)
        at org.bytedeco.javacpp.Loader.load(Loader.java:1212)
        at org.bytedeco.javacpp.Loader.load(Loader.java:1157)
        at org.bytedeco.javacpp.Loader.load(Loader.java:1133)
        at org.bytedeco.tesseract.TessBaseAPI.<clinit>(TessBaseAPI.java:26)
        at org.acejump.tracejump.Reader.<clinit>(Reader.kt:30)
        at org.acejump.tracejump.TraceJump.screenWatcher$lambda-2(TraceJump.kt:60)
        at java.base/java.lang.Thread.run(Thread.java:833)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.UnsatisfiedLinkError: Could not find jnilept in class, module, and library paths.
        at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1705)
        ... 14 more

@saudet
Copy link
Member

saudet commented Nov 20, 2021

No, not yet, but contributions are welcome, see issue #1069.

saudet added a commit that referenced this issue Apr 1, 2022
…1163))

 * Introduce `macosx-arm64` builds for Leptonica and Tesseract ([issue #814](#814))
saudet added a commit that referenced this issue Apr 1, 2022
 * Introduce `macosx-arm64` builds for Leptonica and Tesseract (issue #814)
 * Upgrade presets for zlib 1.2.12
@saudet
Copy link
Member

saudet commented Apr 1, 2022

@breandan I've introduced macosx-arm64 builds for Leptonica and Tesseract!
Please give it a try with the snapshots: http://bytedeco.org/builds/

@breandan
Copy link
Author

breandan commented Apr 1, 2022

Thanks @saudet, much appreciated! Hope you're doing well.

@saudet
Copy link
Member

saudet commented Apr 1, 2022

@breandan BTW, please list what you guys are using at MILA in issue #1069 so we can get some sort of idea of what to prioritize for AI applications like that. Thanks!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants