Skip to main content

JAXB 2.0 - OutOfMemory

15 replies [Last post]
kirillcool
Offline
Joined: 2004-11-17
Points: 0

I have tried to run XJC on a fairly big schema (with a lot of includes), and the result was not good.
With default JVM settings i got OutOfMemory after a few seconds. With Xmx1024m, the memory jumped pretty quickly to 872MB (!!!), and then nothing happened for over 15 minutes (after the "parsing schema..." message). After that i killed the process.

The xsom parsing takes about 3-4 seconds, after which it provide complete (and correct) information on all the schema elements. However, the parsing phase in JAXB 2.0 seems to be stuck for good.

Unfortunately i can not provide the schema as it is our internal company IP, but the size estimation from xsom sample visitor is 7640 lines and 330K total size.

I wonder if the internal tests include schemas of this size.

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
hr_stoyanov
Offline
Joined: 2004-04-01
Points: 0

Kohsuke-,
I would also appreciate the XJC ANT switches for the IRS schemas youused, if you have time.

Thanks
Hristo

kohsuke
Offline
Joined: 2003-06-09
Points: 0

Nothing. It compiles out of the box with the 2.0 RI.

hr_stoyanov
Offline
Joined: 2004-04-01
Points: 0

Thanks Kohsuke-,
In future, you may want to get you IRS test samples from:

http://www.taxadmin.org/

,where IRS posts all their lates schemas. These are easily the hardest/largest schema sets you could test JAXB against.

Thanks,
Hristo

kohsuke
Offline
Joined: 2003-06-09
Points: 0

Thanks for the pointer.

bobc
Offline
Joined: 2005-04-13
Points: 0

Another schema you may want to check out for testing purposes - the GJXDM, a law enforcement XML schema. It's over 8 megs and has killed a number of tools (JAXB 1.x among them). Available here:

http://it.ojp.gov/jxdm/

kohsuke
Offline
Joined: 2003-06-09
Points: 0

An earlier version of this has been in our unit test, actually (when this is still called JXDD.)

I'm going to replace it with JXDM. I made a few fixes to the RI, and then JXDM compiled out of box. I had to set -Xmx128mb though (looking at -verbose:gc, probably -Xmx96mb would be sufficient.)

It generated some 572 source files.

hr_stoyanov
Offline
Joined: 2004-04-01
Points: 0

Koshuke-,
Which of the IRS schemas do you use for your internal tests? I am very interested in (at least):
940/941 - http://www.irs.gov/taxpros/providers/article/0,,id=97753,00.html

In the past 1120 (http://www.irs.gov/taxpros/providers/article/0,,id=97626,00.html)
made me switch to XmlBeans and give up on JAXB 1.x

Thanks,
Hristo

kohsuke
Offline
Joined: 2003-06-09
Points: 0

The folder name is 'efile1120x_20040830'.

Taken from http://www.irs.gov/efile/article/0,,id=128360,00.html

In particular "2004v1.0" at http://www.irs.gov/pub/irs-schema/efile1120x_20040830.zip

sdp198
Offline
Joined: 2005-04-06
Points: 0

I had out of memory problems which went when I removed a line from my binding file:


Not sure why it caused a problem, but it meant I had to work round the name clash it would otherwise have fixed.

kohsuke
Offline
Joined: 2003-06-09
Points: 0

I wonder if you could file an issue to http://jaxb.java.net/issues/ so that we can reproduce & fix the problem.

sdp198
Offline
Joined: 2005-04-06
Points: 0

Yep, I've added one.

kohsuke
Offline
Joined: 2003-06-09
Points: 0

Thanks. I'll look into this.

kohsuke
Offline
Joined: 2003-06-09
Points: 0

In our tests we use schemas like IFX, FpML, and IRS e-file schemas, which are a lot bigger than your 330K. For example the IRS schema is around 6MB in size. They all compile fine with the default memory setting.

I suspect that there's a particular use of a schema construct/feature in your schema that causes a lot of memory consumption in XJC. It would really help me if I can get my hands on the schema.

Is there any chance you could send us the schema privately? If you wrote a XSOM visitor, you might be able to 'scramble' the schema by changing all the tag names and so on, if you really need to do so.

kirillcool
Offline
Joined: 2004-11-17
Points: 0

Kohsuke,

How does the output of xsom visitor help? it's just one big xml with tags that have names that are different from xsd tags. In addition, if the problem lies in includes/imports, it will not help.

I'll think of something, maybe some kind of scrambler for all the relevant schemas that will replace all elements with aNNNNN names using simple SAX echoers.

It will take a few days anyway.

kohsuke
Offline
Joined: 2003-06-09
Points: 0

Internally XJC works off of the XSOM model, so given that you were able to parse it through XSOM, I doubt if it's import/include issue.

I thought you were able to use XSOM dumper to dump back the XML schema, and so I thought that would allow you to scramble XML Schema relatively easily (by tweaking the dumper.)

In any case, if you could find a way for us to reproduce the problem, like a scrambler, that would be greatly appreciated.