Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java: add dataflow-generated models for JDK17 #14919

Merged
merged 11 commits into from
Apr 4, 2024

Conversation

owen-mc
Copy link
Contributor

@owen-mc owen-mc commented Nov 27, 2023

Add dataflow-generated models for JDK17 and update test expectations as needed.

Note that preparatory work was done in:

  • this PR (adding manual models to override dataflow-generated models that didn't combine well with some existing manual models, and also adding neutral summary models for some callables that we don't want summary models for)
  • this PR (making manual neutral summary models block dataflow-generated summary models)
  • this PR (don't generate models that won't be used because the are blocked by a manual model, and also don't generate summary and neutral models for the same callable).
  • this PR (add some neutral models)
  • this PR (add some summary models and some neutral models)
  • this PR (add some neutral models)
  • Java: more manual models
  • Java: Add more neutrals and improve java.net.URL models

Note to reviewers: obviously you can't look through all the generated models. If you want you can do some spot checking to make sure things look okay. But I think it's okay to just look at the other commits and makes sure that the test changes do look reasonable given the new models. If we find that some of the generated models are wrong then we can add manual models for those callables, but that shouldn't stop us from merging these generated models that we believe are significantly correct. If there are improvements to the model generator in future then we can regenerate models for the JDK and benefit from improved models.

@github-actions github-actions bot added the Java label Nov 27, 2023
@owen-mc owen-mc force-pushed the java/jdk-model-autogeneration branch from f1089fd to 37137d5 Compare November 27, 2023 15:15
Copy link
Contributor

github-actions bot commented Nov 27, 2023

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

java

Generated file changes for java

  • Changes to framework-coverage-java.rst:
-    Java Standard Library,``java.*``,10,760,240,80,,9,,,26
+    Java Standard Library,``java.*``,10,4620,240,80,,9,,,26
-    Java extensions,"``javax.*``, ``jakarta.*``",69,688,85,5,4,2,1,1,4
+    Java extensions,"``javax.*``, ``jakarta.*``",69,3257,85,5,4,2,1,1,4
-    Others,"``actions.osgi``, ``antlr``, ``ch.ethz.ssh2``, ``cn.hutool.core.codec``, ``com.alibaba.druid.sql``, ``com.alibaba.fastjson2``, ``com.amazonaws.auth``, ``com.auth0.jwt.algorithms``, ``com.azure.identity``, ``com.esotericsoftware.kryo.io``, ``com.esotericsoftware.kryo5.io``, ``com.fasterxml.jackson.core``, ``com.fasterxml.jackson.databind``, ``com.google.gson``, ``com.hubspot.jinjava``, ``com.jcraft.jsch``, ``com.microsoft.sqlserver.jdbc``, ``com.mitchellbosecke.pebble``, ``com.mongodb``, ``com.opensymphony.xwork2``, ``com.rabbitmq.client``, ``com.sshtools.j2ssh.authentication``, ``com.sun.crypto.provider``, ``com.sun.jndi.ldap``, ``com.sun.net.httpserver``, ``com.sun.net.ssl``, ``com.sun.rowset``, ``com.sun.security.auth.module``, ``com.sun.security.ntlm``, ``com.sun.security.sasl.digest``, ``com.thoughtworks.xstream``, ``com.trilead.ssh2``, ``com.unboundid.ldap.sdk``, ``com.zaxxer.hikari``, ``flexjson``, ``freemarker.cache``, ``freemarker.template``, ``groovy.lang``, ``groovy.text``, ``groovy.util``, ``hudson``, ``io.jsonwebtoken``, ``io.netty.bootstrap``, ``io.netty.buffer``, ``io.netty.channel``, ``io.netty.handler.codec``, ``io.netty.handler.ssl``, ``io.netty.handler.stream``, ``io.netty.resolver``, ``io.netty.util``, ``javafx.scene.web``, ``jenkins``, ``jodd.json``, ``liquibase.database.jvm``, ``liquibase.statement.core``, ``net.schmizz.sshj``, ``net.sf.json``, ``net.sf.saxon.s9api``, ``ognl``, ``okhttp3``, ``org.acegisecurity``, ``org.antlr.runtime``, ``org.apache.commons.codec``, ``org.apache.commons.compress.archivers.tar``, ``org.apache.commons.exec``, ``org.apache.commons.httpclient.util``, ``org.apache.commons.jelly``, ``org.apache.commons.jexl2``, ``org.apache.commons.jexl3``, ``org.apache.commons.lang``, ``org.apache.commons.logging``, ``org.apache.commons.net``, ``org.apache.commons.ognl``, ``org.apache.cxf.catalog``, ``org.apache.cxf.common.classloader``, ``org.apache.cxf.common.jaxb``, ``org.apache.cxf.common.logging``, ``org.apache.cxf.configuration.jsse``, ``org.apache.cxf.helpers``, ``org.apache.cxf.resource``, ``org.apache.cxf.staxutils``, ``org.apache.cxf.tools.corba.utils``, ``org.apache.cxf.tools.util``, ``org.apache.cxf.transform``, ``org.apache.directory.ldap.client.api``, ``org.apache.hadoop.fs``, ``org.apache.hadoop.hive.metastore``, ``org.apache.hadoop.hive.ql.exec``, ``org.apache.hadoop.hive.ql.metadata``, ``org.apache.hc.client5.http.async.methods``, ``org.apache.hc.client5.http.classic.methods``, ``org.apache.hc.client5.http.fluent``, ``org.apache.hive.hcatalog.templeton``, ``org.apache.ibatis.jdbc``, ``org.apache.ibatis.mapping``, ``org.apache.log4j``, ``org.apache.shiro.codec``, ``org.apache.shiro.jndi``, ``org.apache.shiro.mgt``, ``org.apache.sshd.client.session``, ``org.apache.struts.beanvalidation.validation.interceptor``, ``org.apache.struts2``, ``org.apache.tools.ant``, ``org.apache.tools.zip``, ``org.apache.velocity.app``, ``org.apache.velocity.runtime``, ``org.codehaus.cargo.container.installer``, ``org.codehaus.groovy.control``, ``org.dom4j``, ``org.eclipse.jetty.client``, ``org.fusesource.leveldbjni``, ``org.geogebra.web.full.main``, ``org.gradle.api.file``, ``org.hibernate``, ``org.influxdb``, ``org.jdbi.v3.core``, ``org.jenkins.ui.icon``, ``org.jenkins.ui.symbol``, ``org.jooq``, ``org.keycloak.models.map.storage``, ``org.kohsuke.stapler``, ``org.mvel2``, ``org.openjdk.jmh.runner.options``, ``org.owasp.esapi``, ``org.pac4j.jwt.config.encryption``, ``org.pac4j.jwt.config.signature``, ``org.scijava.log``, ``org.slf4j``, ``org.thymeleaf``, ``org.xml.sax``, ``org.xmlpull.v1``, ``org.yaml.snakeyaml``, ``play.libs.ws``, ``play.mvc``, ``ratpack.core.form``, ``ratpack.core.handling``, ``ratpack.core.http``, ``ratpack.exec``, ``ratpack.form``, ``ratpack.func``, ``ratpack.handling``, ``ratpack.http``, ``ratpack.util``, ``retrofit2``, ``sun.jvmstat.perfdata.monitor.protocol.local``, ``sun.jvmstat.perfdata.monitor.protocol.rmi``, ``sun.misc``, ``sun.net.ftp``, ``sun.net.www.protocol.http``, ``sun.security.acl``, ``sun.security.jgss.krb5``, ``sun.security.krb5``, ``sun.security.pkcs``, ``sun.security.pkcs11``, ``sun.security.provider``, ``sun.security.ssl``, ``sun.security.x509``, ``sun.tools.jconsole``",131,10518,893,125,6,22,18,,208
+    Others,"``actions.osgi``, ``antlr``, ``ch.ethz.ssh2``, ``cn.hutool.core.codec``, ``com.alibaba.druid.sql``, ``com.alibaba.fastjson2``, ``com.amazonaws.auth``, ``com.auth0.jwt.algorithms``, ``com.azure.identity``, ``com.esotericsoftware.kryo.io``, ``com.esotericsoftware.kryo5.io``, ``com.fasterxml.jackson.core``, ``com.fasterxml.jackson.databind``, ``com.google.gson``, ``com.hubspot.jinjava``, ``com.jcraft.jsch``, ``com.microsoft.sqlserver.jdbc``, ``com.mitchellbosecke.pebble``, ``com.mongodb``, ``com.opensymphony.xwork2``, ``com.rabbitmq.client``, ``com.sshtools.j2ssh.authentication``, ``com.sun.crypto.provider``, ``com.sun.jndi.ldap``, ``com.sun.net.httpserver``, ``com.sun.net.ssl``, ``com.sun.rowset``, ``com.sun.security.auth.module``, ``com.sun.security.ntlm``, ``com.sun.security.sasl.digest``, ``com.thoughtworks.xstream``, ``com.trilead.ssh2``, ``com.unboundid.ldap.sdk``, ``com.zaxxer.hikari``, ``flexjson``, ``freemarker.cache``, ``freemarker.template``, ``groovy.lang``, ``groovy.text``, ``groovy.util``, ``hudson``, ``io.jsonwebtoken``, ``io.netty.bootstrap``, ``io.netty.buffer``, ``io.netty.channel``, ``io.netty.handler.codec``, ``io.netty.handler.ssl``, ``io.netty.handler.stream``, ``io.netty.resolver``, ``io.netty.util``, ``javafx.scene.web``, ``jdk.internal.access``, ``jdk.internal.org.objectweb.asm``, ``jenkins``, ``jodd.json``, ``liquibase.database.jvm``, ``liquibase.statement.core``, ``net.schmizz.sshj``, ``net.sf.json``, ``net.sf.saxon.s9api``, ``ognl``, ``okhttp3``, ``org.acegisecurity``, ``org.antlr.runtime``, ``org.apache.commons.codec``, ``org.apache.commons.compress.archivers.tar``, ``org.apache.commons.exec``, ``org.apache.commons.httpclient.util``, ``org.apache.commons.jelly``, ``org.apache.commons.jexl2``, ``org.apache.commons.jexl3``, ``org.apache.commons.lang``, ``org.apache.commons.logging``, ``org.apache.commons.net``, ``org.apache.commons.ognl``, ``org.apache.cxf.catalog``, ``org.apache.cxf.common.classloader``, ``org.apache.cxf.common.jaxb``, ``org.apache.cxf.common.logging``, ``org.apache.cxf.configuration.jsse``, ``org.apache.cxf.helpers``, ``org.apache.cxf.resource``, ``org.apache.cxf.staxutils``, ``org.apache.cxf.tools.corba.utils``, ``org.apache.cxf.tools.util``, ``org.apache.cxf.transform``, ``org.apache.directory.ldap.client.api``, ``org.apache.hadoop.fs``, ``org.apache.hadoop.hive.metastore``, ``org.apache.hadoop.hive.ql.exec``, ``org.apache.hadoop.hive.ql.metadata``, ``org.apache.hc.client5.http.async.methods``, ``org.apache.hc.client5.http.classic.methods``, ``org.apache.hc.client5.http.fluent``, ``org.apache.hive.hcatalog.templeton``, ``org.apache.ibatis.jdbc``, ``org.apache.ibatis.mapping``, ``org.apache.log4j``, ``org.apache.shiro.codec``, ``org.apache.shiro.jndi``, ``org.apache.shiro.mgt``, ``org.apache.sshd.client.session``, ``org.apache.struts.beanvalidation.validation.interceptor``, ``org.apache.struts2``, ``org.apache.tools.ant``, ``org.apache.tools.zip``, ``org.apache.velocity.app``, ``org.apache.velocity.runtime``, ``org.codehaus.cargo.container.installer``, ``org.codehaus.groovy.control``, ``org.dom4j``, ``org.eclipse.jetty.client``, ``org.fusesource.leveldbjni``, ``org.geogebra.web.full.main``, ``org.gradle.api.file``, ``org.hibernate``, ``org.influxdb``, ``org.jdbi.v3.core``, ``org.jenkins.ui.icon``, ``org.jenkins.ui.symbol``, ``org.jooq``, ``org.keycloak.models.map.storage``, ``org.kohsuke.stapler``, ``org.mvel2``, ``org.openjdk.jmh.runner.options``, ``org.owasp.esapi``, ``org.pac4j.jwt.config.encryption``, ``org.pac4j.jwt.config.signature``, ``org.scijava.log``, ``org.slf4j``, ``org.thymeleaf``, ``org.w3c.dom``, ``org.xml.sax``, ``org.xmlpull.v1``, ``org.yaml.snakeyaml``, ``play.libs.ws``, ``play.mvc``, ``ratpack.core.form``, ``ratpack.core.handling``, ``ratpack.core.http``, ``ratpack.exec``, ``ratpack.form``, ``ratpack.func``, ``ratpack.handling``, ``ratpack.http``, ``ratpack.util``, ``retrofit2``, ``sun.awt``, ``sun.jvmstat.perfdata.monitor.protocol.local``, ``sun.jvmstat.perfdata.monitor.protocol.rmi``, ``sun.management.spi``, ``sun.misc``, ``sun.net.ftp``, ``sun.net.www.protocol.http``, ``sun.nio.ch``, ``sun.security.acl``, ``sun.security.jgss.krb5``, ``sun.security.krb5``, ``sun.security.pkcs``, ``sun.security.pkcs11``, ``sun.security.provider``, ``sun.security.ssl``, ``sun.security.x509``, ``sun.tools.jconsole``, ``sun.util.logging.internal``",131,10734,893,125,6,22,18,,208
-    Totals,,310,18976,2569,338,16,128,33,1,409
+    Totals,,310,25621,2569,338,16,128,33,1,409
  • Changes to framework-coverage-java.csv:
+ java.applet,,,14,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,14,
- java.awt,1,,3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,,,,,,,,,,3
+ java.awt,1,,5,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,,,,,,,,,2,3
- java.beans,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
+ java.beans,,,193,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,193,
- java.io,51,1,47,,,,,,,,,22,,,,,,,,,,,,,,,29,,,,,,,,,,,,,,,,,,,,,,1,,45,2
+ java.io,51,1,256,,,,,,,,,22,,,,,,,,,,,,,,,29,,,,,,,,,,,,,,,,,,,,,,1,,249,7
- java.lang,38,3,102,,13,,,,,,1,,,,,,,,,,,,8,,,,11,,,4,,,1,,,,,,,,,,,,,,,3,,,59,43
+ java.lang,38,3,759,,13,,,,,,1,,,,,,,,,,,,8,,,,11,,,4,,,1,,,,,,,,,,,,,,,3,,,681,78
+ java.math,,,9,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,9
- java.net,23,3,31,,,,1,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,21,,,,,,,,,,,,,,3,31,
+ java.net,23,3,278,,,,1,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,21,,,,,,,,,,,,,,3,274,4
- java.nio,44,,38,,,,,,,,,5,,,,,,,,,,,,,,,38,,,,,,,,,1,,,,,,,,,,,,,,,38,
+ java.nio,44,,361,,,,,,,,,5,,,,,,,,,,,,,,,38,,,,,,,,,1,,,,,,,,,,,,,,,259,102
+ java.rmi,,,71,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,71,
- java.security,21,,7,,,11,10,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,3,4
+ java.security,21,,543,,,11,10,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,539,4
- java.sql,15,1,2,,,,1,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,4,,9,,,,,,,,,1,,,,2,
+ java.sql,15,1,303,,,,1,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,4,,9,,,,,,,,,1,,,,303,
+ java.text,,,134,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,134,
+ java.time,,,476,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,388,88
- java.util,47,2,529,,,,,,,,,1,,,,,,,,,,,34,,,,2,,,,5,2,,1,2,,,,,,,,,,,,,2,,,49,480
+ java.util,47,2,1218,,,,,,,,,1,,,,,,,,,,,34,,,,2,,,,5,2,,1,2,,,,,,,,,,,,,2,,,704,514
+ javax.accessibility,,,31,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,31,
+ javax.annotation.processing,,,9,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,9,
- javax.crypto,19,,4,,,12,3,,2,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,4,
+ javax.crypto,19,,128,,,12,3,,2,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,128,
- javax.imageio.stream,1,,1,,,,,,,,,,,,,,,,,,,,,,,,1,,,,,,,,,,,,,,,,,,,,,,,,1,
+ javax.imageio,1,,261,,,,,,,,,,,,,,,,,,,,,,,,1,,,,,,,,,,,,,,,,,,,,,,,,261,
+ javax.lang.model.element,,,17,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,17,
+ javax.lang.model.type,,,9,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,9,
+ javax.lang.model.util,,,68,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,68,
- javax.management,2,,1,,,,,,,,,,,,,,,,,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
+ javax.management,2,,799,,,,,,,,,,,,,,,,,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,798,1
- javax.naming,7,,1,,,,,,,,,,,,,,,,,6,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
+ javax.naming,7,,324,,,,,,,,,,,,,,,,,6,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,318,6
- javax.net.ssl,4,,,,,,2,,,,,,,,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
+ javax.net,4,,86,,,,2,,,,,,,,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,86,
- javax.print.attribute.standard,2,,,,,,,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
+ javax.print,2,,100,,,,,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,100,
+ javax.rmi.ssl,,,7,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,7,
- javax.script,1,,,,,,,,,,,,,,,,,,,,,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,
+ javax.script,1,,42,,,,,,,,,,,,,,,,,,,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,42,
- javax.security.auth.callback,1,,,,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
- javax.security.auth.kerberos,6,,,,,4,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
+ javax.security.auth,7,,137,,,4,3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,137,
+ javax.security.cert,,,5,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,5,
+ javax.security.sasl,,,28,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,28,
+ javax.smartcardio,,,30,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,30,
+ javax.sound.midi,,,29,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,29,
+ javax.sound.sampled,,,66,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,66,
- javax.sql,7,,,,,,4,3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
+ javax.sql,7,,63,,,,4,3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,63,
+ javax.tools,,,31,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,31,
+ javax.transaction.xa,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
+ javax.xml.catalog,,,11,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,10,1
+ javax.xml.crypto,,,126,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,126,
+ javax.xml.datatype,,,4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,4,
+ javax.xml.namespace,,,10,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,10,
+ javax.xml.parsers,,,18,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,18,
+ javax.xml.stream,,,36,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,36,
- javax.xml.transform,2,,6,,,,,,,,,,,,,,,,,,,,,,,,1,,,,,,,,,,,,,,,,,1,,,,,,,6,
+ javax.xml.transform,2,,89,,,,,,,,,,,,,,,,,,,,,,,,1,,,,,,,,,,,,,,,,,1,,,,,,,89,
+ javax.xml.validation,,,5,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,5,
- javax.xml.xpath,3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,3,,,,,,,,,
+ javax.xml.xpath,3,,12,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,3,,,,,,,,12,
+ jdk.internal.access,,,136,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,136,
+ jdk.internal.org.objectweb.asm,,,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2,
+ org.w3c.dom,,,57,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,56,1
- org.xml.sax,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
+ org.xml.sax,,,4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,4,
+ sun.awt,,,5,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,5,
+ sun.management.spi,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
+ sun.nio.ch,,,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2,
- sun.security.krb5,9,,,,,3,6,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
+ sun.security.krb5,9,,7,,,3,6,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,7,
+ sun.util.logging.internal,,,3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,3,

@owen-mc owen-mc force-pushed the java/jdk-model-autogeneration branch from 37137d5 to c0db253 Compare December 13, 2023 15:53
@owen-mc owen-mc force-pushed the java/jdk-model-autogeneration branch from a17ba00 to 5661caf Compare January 4, 2024 12:21
@owen-mc owen-mc force-pushed the java/jdk-model-autogeneration branch 5 times, most recently from e2bd8d9 to 713ebed Compare January 14, 2024 20:49
@owen-mc owen-mc marked this pull request as ready for review January 14, 2024 22:08
@owen-mc owen-mc requested a review from a team as a code owner January 14, 2024 22:08
@owen-mc owen-mc force-pushed the java/jdk-model-autogeneration branch from cb6455c to 2d172b0 Compare February 29, 2024 13:18
@owen-mc owen-mc force-pushed the java/jdk-model-autogeneration branch 3 times, most recently from 9187679 to 7941085 Compare March 6, 2024 16:25
atorralba added a commit to atorralba/codeql that referenced this pull request Mar 13, 2024
This is similar to github#15766, in the sense that it adds neutral models to prevent the model generator from generating summaries for them. These models were spotted while evaluating github#14919.
@owen-mc owen-mc force-pushed the java/jdk-model-autogeneration branch from 7941085 to 0fb42a1 Compare March 19, 2024 15:12
---
category: minorAnalysis
---
* About 15,000 summary models and neutral summary models for the JDK that were generated using data flow have been added. This may lead to new results being reported.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally use alerts instead of results in most communications.

owen-mc added 2 commits March 27, 2024 12:15
The command line was:
python3 /Users/owen-mc/workspace/codeql-home/codeql/java/ql/src/utils/modelgenerator/GenerateFlowModel.py /Users/owen-mc/db/java/openjdk17 --with-summaries --with-neutrals
Manual models always take precedence over generated models, so there is
no point in keeping the generated models.

These APIs happened to have been modelled between model generation and
merging this PR.
@owen-mc owen-mc force-pushed the java/jdk-model-autogeneration branch from ab529f0 to 71347ce Compare March 27, 2024 12:21
@owen-mc owen-mc changed the title Java: openjdk model autogeneration Java: add dataflow-generated models for JDK17 Mar 27, 2024
atorralba
atorralba previously approved these changes Mar 30, 2024
owen-mc added 9 commits April 1, 2024 13:59
Manual models always take precedence over generated models, so there is
no point in keeping the generated models.

These manual models were deliberately written to take precedence over
the corresponding df-gen models.
This line is added because `FileOutputStream`'s constructor is now modeled as propagating taint, not just as a sink.
| p;PrivateFlowViaPublicInterface$SPI;true;openStream;();;Argument[this];ReturnValue;taint;df-generated |
@owen-mc owen-mc force-pushed the java/jdk-model-autogeneration branch from 71347ce to fdafaa2 Compare April 1, 2024 13:04
@owen-mc
Copy link
Contributor Author

owen-mc commented Apr 1, 2024

Rebasing on #16062 made a test fail. I have fixed that and also updated the commits which delete generated models which have been deliberately/incidentally modeled since I generated models. This is ready to review and hopefully merge soon.

Copy link
Contributor

@smowton smowton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looked at evaluation results, which appear positive

@owen-mc owen-mc merged commit e10333b into main Apr 4, 2024
16 checks passed
@owen-mc owen-mc deleted the java/jdk-model-autogeneration branch April 4, 2024 20:12
@coadaflorin
Copy link
Contributor

Is this going to be in 2.17.1? If so, I'll prep some wording for that release changelog.

@owen-mc
Copy link
Contributor Author

owen-mc commented Apr 5, 2024

I believe so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants