Get validate.sh working again

[Klortho]

This is the script to do validation against the schematrons offline.

It should be very easy to get it to work -- it just means adding catalog files to find the right DTDs. That means we need a master catalog for each nlm-dtd and niso-jats, and a master master catalog that imports each of those. These could replace dtds.yaml, maybe.

[hubgit]

Instead of having a master catalog for each DTD, it's easier to just map URL prefixes to local directories containing a mirror of the DTDs.

[Klortho]

Well, we already have a set of flattened DTDs, and a dtds.yaml file that maps all the public identifiers. I'm hesitant to rely on the system ids -- in that past, people have said that they explicitly avoided recommending putting the full NLM URI into the system identifier, because they are worried about our servers getting overloaded; and referring to the problem with the HTML DTDs that W3C got themselves into.

[hubgit]

If the system identifier isn't a URL, then there's no point in having it there at all - it's useless outside the context of a specific application.

Having the system identifier as a URL, however, is very useful, as it allows remote paths to be mapped to local paths.

[jeffbeckncbi]

We see a lot of content coming to us with a PUBLIC ID and the DTD filename for the SYSTEM ID. And sometimes the PUBLIC ID is just wrong (they are notoriously hard to type).

It would cause much less frustration if we mapped SYSTEM IDs of just the DTD filename to our local copies of the DTD. Anything else we would be ok to ignore.

But we should be able to resolve

[Klortho]

We talked about this at the telecon a little bit yesterday. Rather than just getting validate.sh working, we really need to create a stand-alone validator that users can download and run on their own systems. I have in mind something that could run either as a web service, that we can run on our AMI, or could be downloaded. Ideas welcome.