Well a bit of Googling turned up that Ant looks for an environment variable called ANT_OPTS which is use to set Java parameters. Just set the environment variable and off you go. So I added the following to increase the heap size:
That sets the maximum heap size to 256 Megabytes. It solved my problem as the XSLT transform topped out at about 220Meg. So if you ever need to increase the size of the Ant JVM, now you know how.