-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Java: Simple support for Ratpack HTTP Framework #4991
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Java: Simple support for Ratpack HTTP Framework #4991
Conversation
java/ql/src/semmle/code/java/frameworks/ratpack/RatpackHttp.qll
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add some tests
java/ql/src/semmle/code/java/frameworks/ratpack/RatpackHttp.qll
Outdated
Show resolved
Hide resolved
java/ql/src/semmle/code/java/frameworks/ratpack/RatpackHttp.qll
Outdated
Show resolved
Hide resolved
I was looking into doing this, but there didn't look like there were any sample/examples for spring, nor apache. Do you have a good suggestion to base these tests upon? |
You can identify existing tests using a particularly library by looking for the All existing library stubs can be found here: https://github.com/github/codeql/tree/main/java/ql/test/stubs New ones are created by stripping function bodies, essentially producing Java files that document the library's public interface but not method definitions. |
@JLLeitschuh do you intend to pursue this? |
I do, I just haven't had the time yet |
295e866
to
e516537
Compare
} | ||
|
||
/** | ||
* Methods on `ratpack.http.TypedData` that return user tainted data. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Check: you're sure TypedData, which sounds pretty generic, can always be assumed to come from an untrusted source?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to @ldaley, one of the Ratpack authors, almost always.
The actual source of the data would be Request::getBody
, where TypedData
is where this input gets pulled from the body. However, there are cases in user libraries where UploadedFile
(a supertype of TypedData
) is instantiated inside a Jackson deserialized object, which isn't otherwise easily traceable.
Now that you're asking, it would probably be better to use TypedData
as a taint passthrough and make sure that getBody
is captured as a source instead. 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I refactored this to use getBody
as the remote flow source and calls on TypedData
to just be taint flow.
106ce24
to
eb27f40
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me; running tests and passing to @aschackmull for final review as this targets the main query pack.
Given that, it will also need a change note @JLLeitschuh
Check failures: File "ql/java/ql/test/stubs/ratpack-1.9.x/ratpack/core/handling/Context.java" contains a non-ASCII character at the location marked with
File "ql/java/ql/test/stubs/ratpack-1.9.x/ratpack/core/handling/Handler.java" contains a non-ASCII character at the location marked with
File "ql/java/ql/test/stubs/ratpack-1.9.x/ratpack/exec/Promise.java" contains a non-ASCII character at the location marked with
File "ql/java/ql/test/stubs/ratpack-1.9.x/ratpack/exec/registry/Registry.java" contains a non-ASCII character at the location marked with
File "ql/java/ql/test/stubs/ratpack-1.9.x/ratpack/func/Predicate.java" contains a non-ASCII character at the location marked with
File "ql/java/ql/test/stubs/ratpack-1.9.x/ratpack/func/Function.java" contains a non-ASCII character at the location marked with
|
QLDoc checks fail because file |
Also the use of type-variables in signatures is apparently going away soon; use the variable's lower bound (usually Object, but String for |
Huh? You mean upper bound, right? A lower bound of |
Apparently so, yes. It appears I've been visualising the Java type hierarchy the wrong way up for about 5 years. |
fefbd2d
to
9518182
Compare
Currently this test fails, I'm wondering if anyone has any insights into why and how I can fix it. void test9() {
String tainted = taint();
Promise
.value(tainted)
.apply(Resource::identity)
.then(value -> {
sink(value); //$hasTaintFlow // passes
});
Promise
.value("potato")
.apply(Resource::identity)
.then(value -> {
sink(value); // no taints flow // fails here
});
}
public static Promise<String> identity(Promise<String> input) {
return input.map(i -> i);
} |
A quick guess is that it has to do with call contexts getting thrown away (and that's something we have to do due to how you are modelling lambda flow, and part of the reason why we want to do it differently). |
To elaborate: Every time you make an additional taint step that jumps from one callable to another, then the call context needs to be discarded (as it doesn't make sense for the new callable). |
I think that this "stuck" tainted method issue is probably intractable with my current approach. I think for now my best approach is to simply drop support of methods like |
332d23f
to
5a2bdc9
Compare
@aschackmull @smowton this is now ready for another review round. 🙂 |
Signed-off-by: Jonathan Leitschuh <[email protected]>
4ba9d75
to
db2892b
Compare
java/ql/lib/semmle/code/java/frameworks/ratpack/RatpackExec.qll
Outdated
Show resolved
Hide resolved
java/ql/lib/semmle/code/java/frameworks/ratpack/RatpackExec.qll
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some minor comments, otherwise looks good to me.
Should be resolved. 🙂 |
|
Signed-off-by: Jonathan Leitschuh <[email protected]>
Resolved the non-ASCII character issue |
There are more non-ascii chars. In particular, the two mentioned in the error message I quoted are still there. |
Signed-off-by: Jonathan Leitschuh <[email protected]>
00c6257
to
5eb2839
Compare
Click to show differences in coveragejavaGenerated file changes for java
- Others,"``cn.hutool.core.codec``, ``com.esotericsoftware.kryo.io``, ``com.esotericsoftware.kryo5.io``, ``com.fasterxml.jackson.core``, ``com.fasterxml.jackson.databind``, ``com.opensymphony.xwork2.ognl``, ``com.unboundid.ldap.sdk``, ``flexjson``, ``groovy.lang``, ``groovy.util``, ``jodd.json``, ``net.sf.saxon.s9api``, ``ognl``, ``org.apache.commons.codec``, ``org.apache.commons.jexl2``, ``org.apache.commons.jexl3``, ``org.apache.commons.ognl``, ``org.apache.directory.ldap.client.api``, ``org.apache.ibatis.jdbc``, ``org.apache.shiro.codec``, ``org.apache.shiro.jndi``, ``org.codehaus.groovy.control``, ``org.dom4j``, ``org.hibernate``, ``org.jooq``, ``org.mvel2``, ``org.xml.sax``, ``org.xmlpull.v1``, ``play.mvc``",7,28,151,,,,14,18,,
+ Others,"``cn.hutool.core.codec``, ``com.esotericsoftware.kryo.io``, ``com.esotericsoftware.kryo5.io``, ``com.fasterxml.jackson.core``, ``com.fasterxml.jackson.databind``, ``com.opensymphony.xwork2.ognl``, ``com.unboundid.ldap.sdk``, ``flexjson``, ``groovy.lang``, ``groovy.util``, ``jodd.json``, ``net.sf.saxon.s9api``, ``ognl``, ``org.apache.commons.codec``, ``org.apache.commons.jexl2``, ``org.apache.commons.jexl3``, ``org.apache.commons.ognl``, ``org.apache.directory.ldap.client.api``, ``org.apache.ibatis.jdbc``, ``org.apache.shiro.codec``, ``org.apache.shiro.jndi``, ``org.codehaus.groovy.control``, ``org.dom4j``, ``org.hibernate``, ``org.jooq``, ``org.mvel2``, ``org.xml.sax``, ``org.xmlpull.v1``, ``play.mvc``, ``ratpack.core.form``, ``ratpack.core.handling``, ``ratpack.core.http``, ``ratpack.exec``, ``ratpack.form``, ``ratpack.func``, ``ratpack.handling``, ``ratpack.http``, ``ratpack.util``",39,99,151,,,,14,18,,
- Totals,,143,5261,408,13,6,10,107,33,1,66
+ Totals,,175,5332,408,13,6,10,107,33,1,66
- com.fasterxml.jackson.databind,,,5,,,,,,,,,,,,,,,,,,,,,5,
+ com.fasterxml.jackson.databind,,,6,,,,,,,,,,,,,,,,,,,,,6,
+ ratpack.core.form,,,3,,,,,,,,,,,,,,,,,,,,,3,
+ ratpack.core.handling,,6,4,,,,,,,,,,,,,,,,,,,,6,4,
+ ratpack.core.http,,10,10,,,,,,,,,,,,,,,,,,,,10,10,
+ ratpack.exec,,,26,,,,,,,,,,,,,,,,,,,,,,26
+ ratpack.form,,,3,,,,,,,,,,,,,,,,,,,,,3,
+ ratpack.func,,,5,,,,,,,,,,,,,,,,,,,,,,5
+ ratpack.handling,,6,4,,,,,,,,,,,,,,,,,,,,6,4,
+ ratpack.http,,10,10,,,,,,,,,,,,,,,,,,,,10,10,
+ ratpack.util,,,5,,,,,,,,,,,,,,,,,,,,,,5 |
Oof... I cleaned up the |
There are some more non-ascii quotes in Promise.java. |
Signed-off-by: Jonathan Leitschuh <[email protected]>
Click to show differences in coveragejavaGenerated file changes for java
- Others,"``cn.hutool.core.codec``, ``com.esotericsoftware.kryo.io``, ``com.esotericsoftware.kryo5.io``, ``com.fasterxml.jackson.core``, ``com.fasterxml.jackson.databind``, ``com.opensymphony.xwork2.ognl``, ``com.unboundid.ldap.sdk``, ``flexjson``, ``groovy.lang``, ``groovy.util``, ``jodd.json``, ``net.sf.saxon.s9api``, ``ognl``, ``org.apache.commons.codec``, ``org.apache.commons.jexl2``, ``org.apache.commons.jexl3``, ``org.apache.commons.ognl``, ``org.apache.directory.ldap.client.api``, ``org.apache.ibatis.jdbc``, ``org.apache.shiro.codec``, ``org.apache.shiro.jndi``, ``org.codehaus.groovy.control``, ``org.dom4j``, ``org.hibernate``, ``org.jooq``, ``org.mvel2``, ``org.xml.sax``, ``org.xmlpull.v1``, ``play.mvc``",7,28,151,,,,14,18,,
+ Others,"``cn.hutool.core.codec``, ``com.esotericsoftware.kryo.io``, ``com.esotericsoftware.kryo5.io``, ``com.fasterxml.jackson.core``, ``com.fasterxml.jackson.databind``, ``com.opensymphony.xwork2.ognl``, ``com.unboundid.ldap.sdk``, ``flexjson``, ``groovy.lang``, ``groovy.util``, ``jodd.json``, ``net.sf.saxon.s9api``, ``ognl``, ``org.apache.commons.codec``, ``org.apache.commons.jexl2``, ``org.apache.commons.jexl3``, ``org.apache.commons.ognl``, ``org.apache.directory.ldap.client.api``, ``org.apache.ibatis.jdbc``, ``org.apache.shiro.codec``, ``org.apache.shiro.jndi``, ``org.codehaus.groovy.control``, ``org.dom4j``, ``org.hibernate``, ``org.jooq``, ``org.mvel2``, ``org.xml.sax``, ``org.xmlpull.v1``, ``play.mvc``, ``ratpack.core.form``, ``ratpack.core.handling``, ``ratpack.core.http``, ``ratpack.exec``, ``ratpack.form``, ``ratpack.func``, ``ratpack.handling``, ``ratpack.http``, ``ratpack.util``",39,99,151,,,,14,18,,
- Totals,,143,5261,408,13,6,10,107,33,1,66
+ Totals,,175,5332,408,13,6,10,107,33,1,66
- com.fasterxml.jackson.databind,,,5,,,,,,,,,,,,,,,,,,,,,5,
+ com.fasterxml.jackson.databind,,,6,,,,,,,,,,,,,,,,,,,,,6,
+ ratpack.core.form,,,3,,,,,,,,,,,,,,,,,,,,,3,
+ ratpack.core.handling,,6,4,,,,,,,,,,,,,,,,,,,,6,4,
+ ratpack.core.http,,10,10,,,,,,,,,,,,,,,,,,,,10,10,
+ ratpack.exec,,,26,,,,,,,,,,,,,,,,,,,,,,26
+ ratpack.form,,,3,,,,,,,,,,,,,,,,,,,,,3,
+ ratpack.func,,,5,,,,,,,,,,,,,,,,,,,,,,5
+ ratpack.handling,,6,4,,,,,,,,,,,,,,,,,,,,6,4,
+ ratpack.http,,10,10,,,,,,,,,,,,,,,,,,,,10,10,
+ ratpack.util,,,5,,,,,,,,,,,,,,,,,,,,,,5 |
Looks like there's at least one more in Promise.java (on the same line as the one you just removed). |
Signed-off-by: Jonathan Leitschuh <[email protected]>
I'm truly pulling my hair out over these. The VS-code "find" feature isn't doing a good job finding all of the non-ascii quotes. I think I got the last one finally. 😫 |
Click to show differences in coveragejavaGenerated file changes for java
- Others,"``cn.hutool.core.codec``, ``com.esotericsoftware.kryo.io``, ``com.esotericsoftware.kryo5.io``, ``com.fasterxml.jackson.core``, ``com.fasterxml.jackson.databind``, ``com.opensymphony.xwork2.ognl``, ``com.unboundid.ldap.sdk``, ``flexjson``, ``groovy.lang``, ``groovy.util``, ``jodd.json``, ``net.sf.saxon.s9api``, ``ognl``, ``org.apache.commons.codec``, ``org.apache.commons.jexl2``, ``org.apache.commons.jexl3``, ``org.apache.commons.ognl``, ``org.apache.directory.ldap.client.api``, ``org.apache.ibatis.jdbc``, ``org.apache.shiro.codec``, ``org.apache.shiro.jndi``, ``org.codehaus.groovy.control``, ``org.dom4j``, ``org.hibernate``, ``org.jooq``, ``org.mvel2``, ``org.xml.sax``, ``org.xmlpull.v1``, ``play.mvc``",7,28,151,,,,14,18,,
+ Others,"``cn.hutool.core.codec``, ``com.esotericsoftware.kryo.io``, ``com.esotericsoftware.kryo5.io``, ``com.fasterxml.jackson.core``, ``com.fasterxml.jackson.databind``, ``com.opensymphony.xwork2.ognl``, ``com.unboundid.ldap.sdk``, ``flexjson``, ``groovy.lang``, ``groovy.util``, ``jodd.json``, ``net.sf.saxon.s9api``, ``ognl``, ``org.apache.commons.codec``, ``org.apache.commons.jexl2``, ``org.apache.commons.jexl3``, ``org.apache.commons.ognl``, ``org.apache.directory.ldap.client.api``, ``org.apache.ibatis.jdbc``, ``org.apache.shiro.codec``, ``org.apache.shiro.jndi``, ``org.codehaus.groovy.control``, ``org.dom4j``, ``org.hibernate``, ``org.jooq``, ``org.mvel2``, ``org.xml.sax``, ``org.xmlpull.v1``, ``play.mvc``, ``ratpack.core.form``, ``ratpack.core.handling``, ``ratpack.core.http``, ``ratpack.exec``, ``ratpack.form``, ``ratpack.func``, ``ratpack.handling``, ``ratpack.http``, ``ratpack.util``",39,99,151,,,,14,18,,
- Totals,,143,5261,408,13,6,10,107,33,1,66
+ Totals,,175,5332,408,13,6,10,107,33,1,66
- com.fasterxml.jackson.databind,,,5,,,,,,,,,,,,,,,,,,,,,5,
+ com.fasterxml.jackson.databind,,,6,,,,,,,,,,,,,,,,,,,,,6,
+ ratpack.core.form,,,3,,,,,,,,,,,,,,,,,,,,,3,
+ ratpack.core.handling,,6,4,,,,,,,,,,,,,,,,,,,,6,4,
+ ratpack.core.http,,10,10,,,,,,,,,,,,,,,,,,,,10,10,
+ ratpack.exec,,,26,,,,,,,,,,,,,,,,,,,,,,26
+ ratpack.form,,,3,,,,,,,,,,,,,,,,,,,,,3,
+ ratpack.func,,,5,,,,,,,,,,,,,,,,,,,,,,5
+ ratpack.handling,,6,4,,,,,,,,,,,,,,,,,,,,6,4,
+ ratpack.http,,10,10,,,,,,,,,,,,,,,,,,,,10,10,
+ ratpack.util,,,5,,,,,,,,,,,,,,,,,,,,,,5 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎉
Adds some early & very simple support for the Ratpack HTTP framework.
https://ratpack.io/