JsonReader consumes/skips-over trailing content in input stream

[jon-hanson]

The JsonReader seems to be greedy, in that it consumes any content after the first JSON value it finds in the input stream. E.g., in the following example, the trailing " 0" content is skipped over.

    public static void main(String[] args) throws IOException {
        StringReader sr = new StringReader("[] 0"); // Note, string has trailing content.
        JsonReader jr = Json.createReader(sr);
        JsonStructure js = jr.read();
        if (sr.read() != -1) {
            throw null;
        }
        System.out.println(js);
    }

Is this intentional? It would seem more logical to either not consume the trailing content, or to throw a parsing exception.

[niks]

Its the JsonTokenizer that is greedily filling its buffer. This makes it impossible to detect additional data that should not be there.

[jon-hanson]

Which is a bug, do you agree?

[niks]

I'm not sure if this has to be classified as bug. It's rather undefined behavior as the specification does not make any guarantees. It actually doesn't say anything about how many bytes/characters are read.

But this behavior was definitely unexpected by me. And it prevents several use cases where someone wants to read additional data from the same stream. As there is also no information about how many bytes/character have been read, it is also not trivial to work around this by using a BufferedReader/BufferedInputStream with mark() and reset().

[jon-hanson]

Regardless of whether the spec does or doesn't make any guarantees, this behaviour is almost certainly not by design and seems to be, as you describe, an unanticipated side-effect of the implementation of JsonTokenizer It also caused me issues, to the extent that I ended up ditching this parser and writing my own. Given that it is problematic and (most likely) unintended, I would class it as a bug.