Generator doesn't generate method consistently

[mattleibow]

I am not sure where the issue lies - or even what this issue really is. I have noticed that sometimes the generator skips a single method. Just one - sometimes.

I am binding the Kotlin stdlib, so this may be something to do with Kotlin's generated Java code. But, this only happens sometimes, and a rebuild usually corrects it. Not just a /t:rebuild, but a trigger CI again type of rebuild.

The PR that I notice this on is: xamarin/XamarinComponents#678. I am using cake, but I believe the effective command line for MSBuild is:

msbuild /p:Configuration=Release /v:m /bl /t:Pack ./generated/Xamarin.Kotlin.sln

Nothing too special, but the build is not consistent. For some reason, on some builds, a single method is missing:

class ArraysKt___ArraysJvmKt {
     public static final <T> void sort(@NotNull Object[] $this$sort) { ... }
}

This method does exist when I decompile the jar, and it looks to be pretty simple. There is a neighbor method that is the same with two extra int parameters and that is always generated.

I have several builds that are the same commit, but have just that method missing:

• This build has the method in the macOS build, but missing from Windows: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=3115170&view=artifacts
• This build has the method in both the macOS and Windows: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=3115202&view=artifacts

This build is a new commit, but I also zip up the obj folders:

• This build has the method missing from BOTH macOS and Windows: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=3115805&view=artifacts
• This build has the method back again: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=3115830&view=artifacts

A new commit:

• The method is missing just from Windows: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=3115883&view=artifacts

I have no idea what is really going on here since the way machines work is that if the same input is provided to the same machine, the output must be the same - even if it is wrong.

There is one thing that I noticed that MAY be related. I have a Kotlin/Java tool that I run after the API XML is generated to create a Transforms xml file to rename various Kotlin-isms (#464) This tool used to work fine and I have released a few bindings already. But suddenly, it started giving errors that Kotlin could not locate members. All I was doing was using Java reflection to load a member, and then using Kotlin to get the Kotlin version so I could read metadata. This was also not consistent - some builds passed, then some gave errors. You can actually see this in my first PR.

• This build works fine and was green: Improve Kotlin Binding Support xamarin/XamarinComponents#685
• This build failed after the PR was merged: xamarin/XamarinComponents@238ae5c

But that is not all. I eventually just merged as is with a rule to skip that ONE member that wouldn't be consistent: xamarin/XamarinComponents#689 (comment)

[mattleibow]

More info: in this case, I have to use jar2xml because of #464. And the method goes missing right from the start in the api.xml

[Redth]

Does the java version change between builds?

[mattleibow]

Java version? If you mean the .jar file I am binding, then no. If you are talking the actual java.exe, then this is on CI, so it shouldn't be changing.

The version numbers appear to be the same in the binlogs...

[jpobst]

Are you using jar2xml because of #466? If so, I committed a fix for that which should be in 16.4 P2 if you want to try that out. I don't think we're going to be investing much in jar2xml, all the Kotlin support I'm adding is directly in class-parse

[mattleibow]

Just found another library that is missing a few methods.

Trying to bind:
https://search.maven.org/artifact/com.squareup.okhttp3/okhttp/4.0.0/jar

I see the parse method is missing. Java:

@JvmStatic
@NotNull
public static final CacheControl parse(@NotNull Headers headers) {
    return Companion.parse(headers);
}

It is totally missing from the api.xml.

One thing that I notice is that it is a static method. All the others that appear are not. Not sure if that means anything.

BUT! It is there in the api.xml.class-parse!

I have zipped up both files: apis.zip
binlog: msbuild.binlog.zip

Looking closer, I think the Headers type is missing. It is there in the class-parse file, and all I can see is this error from generator.exe:

Error while processing type '[Class] okhttp3.Headers': Type 'kotlin.jvm.internal.markers.KMappedMarker' was not found.

That type is part of a binding dll, but the jar is not embedded - not sure if that makes a difference.

[jpobst]

generator will not generate C# code if it cannot resolve the types referenced. So in this case, kotlin.jvm.internal.markers.KMappedMarker is not in the Kotlin Stdlib nuget, so we remove the okhttp3.Headers type:

Error while processing type '[Class] okhttp3.Headers': Type 'kotlin.jvm.internal.markers.KMappedMarker' was not found.

Then because the parse (Headers headers) method require the Headers type it gets removed as well.

[mattleibow]

I added the missing types into the binding and the headers appeared again.

Waiting on this PR and then everyone will get all their types: xamarin/XamarinComponents#718

Just an FYI for those following along: #371

[jpobst]

I think this is a duplicate of #543, which is fixed for 16.5. Feel free to reopen if this re-occurs.