The latest version of the source code in this article is here. By the time you're reading this I probably updated a couple minor things.
This is about reading XM (FastTracker 2) files in Java. I am not writing a player, now or likely ever. This is because the goal is to eventually write an XM converter. Also there are many perfectly good ways to play XM music.
I've written a few Sega Genesis demos. They aren't great. They exist to see if I could pull them off. I didn't learn how to program audio on the Sega Genesis though. I used a library called Echo which works nicely. I don't think I'll write a sound driver but I've done a lot of things I didn't think I would. Until then I'm sticking with Echo.
Another developer wrote a program to convert XM files to the format used by Echo. It works very well but I can only run it in a Windows virtual machine. This is certainly a "me problem". Even if it's not, I'd like this conversion to be part of the build tools I wrote. So I also have to figure out how to convert XM files to Echo format to achieve this. The bonus is then I'm halfway to supporting other formats like S3M, IT, and MOD. Why not MIDI too? Maybe those are all really difficult which is why only XM conversion exists today. Guess I'll find out the hard way.
Also most of the XM conversions I've done didn't turn out so awesome. The best was in a throwaway demo I wrote in an hour or two. The main flaw is I'm relying on the instrument samples included with Echo. They are good, please don't interpret anything in this article as a critique of Echo. They don't always match up well with the samples in the source XM being converted. I'd like to understand how samples are stored in XM files and maybe even figure out how to convert them to something usable with Echo. Then I could make more accurate conversions of XM files.
That all sounds complicated. Today, let's read an XM file and figure the rest out later.
This is the reference for the XM file format, although I think I found a typo in one field length (2 in the document vs 22 IRL).
Here's how the file is structured:
It took me longer to understand the file structure than it should have. It's not very complicated and I overthought it.
OK, let's code this.
First, a couple tiny code snippets that are referenced frequently. I had this in a different project originally:
public enum Endianness{
LITTLE_ENDIAN,
BIG_ENDIAN;
@Override
public String toString(){
if(this==LITTLE_ENDIAN){return("Little-endian");}
if(this==BIG_ENDIAN){return("Big-endian");}
return(this.name());
}
}
And a couple things to convert arrays of bytes to integers:
public static int byteArrayToInt(byte[] b,Endianness byteOrder){
return(byteArrayToInt(b,0,b.length,byteOrder,true));
}
public static int byteArrayToInt(byte[] b,int startOffset,int length,Endianness byteOrder,boolean unsigned){
int value=0;
for(int i=0;i<length;i++){
int bi=b[i+startOffset];
if(unsigned){bi=bi&0xFF;}
if(byteOrder==Endianness.LITTLE_ENDIAN){
value+=bi*(Math.pow(MAX_UNSIGNED_BYTE,i));
}else if(byteOrder==Endianness.BIG_ENDIAN){
value+=bi*(Math.pow(MAX_UNSIGNED_BYTE,(length-i-1)));
}
}
return(value);
}
Let's setup a bunch of constants before writing anything else, even though these won't be used until a little later:
public abstract class XMConstants{
public static abstract class Offsets{
public static int PATTERN_NUM_ROWS=5;
public static int PATTERN_DATA_SIZE=PATTERN_NUM_ROWS+FieldLengths.PATTERN_NUM_ROWS;
public static int INSTRUMENT_NAME=4;
public static int INSTRUMENT_TYPE=INSTRUMENT_NAME+FieldLengths.INSTRUMENT_NAME;
public static int INSTRUMENT_NUM_SAMPLES=INSTRUMENT_TYPE+FieldLengths.INSTRUMENT_TYPE;
}
public static abstract class FieldLengths{
public static int HEADER_ID=17;
public static int HEADER_MODULENAME=20;
public static int HEADER_TRACKERNAME=20;
public static int HEADER_VERSION=2;
public static int HEADER_SIZE=4;
public static int HEADER_SONG_LENGTH=2;
public static int HEADER_SONG_RESTART=2;
public static int HEADER_SONG_NUM_CHANNELS=2;
public static int HEADER_SONG_NUM_PATTERNS=2;
public static int HEADER_SONG_NUM_INSTRUMENTS=2;
public static int HEADER_SONG_FLAGS=2;
public static int HEADER_SONG_TEMPO=2;
public static int HEADER_SONG_BPM=2;
public static int HEADER_SONG_PATTERN_ORDER=256;
public static int PATTERN_HEADER=4;
public static int PATTERN_NUM_ROWS=2;
public static int PATTERN_DATA_SIZE=2;
public static int INSTRUMENT_HEADER_SIZE=4;
public static int INSTRUMENT_NAME=22;
public static int INSTRUMENT_TYPE=1;
public static int INSTRUMENT_NUM_SAMPLES=2;
public static int INSTRUMENT_SAMPLEHEADER_SIZE=4;
public static int INSTRUMENT_SAMPLENUMBER_SIZE=96;
public static int INSTRUMENT_ENVELOPEPOINTS_SIZE=48;
public static int INSTRUMENT_VOLUMEFADEOUT_SIZE=2;
public static int INSTRUMENT_RESERVED_SIZE=22;
public static int SAMPLE_HEADER_LENGTH=4;
public static int SAMPLE_HEADER_LOOPSTART=4;
public static int SAMPLE_HEADER_LOOPLENGTH=4;
public static int SAMPLE_HEADER_NAME=22;
}
}
The XMFile class itself is very small because all the individual fields are in other classes.
public class XMFile{
/*
* The header - track and song info
*/
private XMHeader header;
public void setHeader(XMHeader header{this.header=header;}
public XMHeader getHeader(){return(this.header);}
/*
* The patterns
*/
private List<XMPattern> patterns;
public void setPatterns(List<XMPattern> patterns){this.patterns=patterns;}
public List<XMPattern> getPatterns(){return(this.patterns);}
/*
* The Instruments (which includes samples)
*/
private List<XMInstrument> instruments;
public void setInstruments(List<XMInstrument> instruments){this.instruments=instruments;}
public List<XMInstrument> getInstruments(){return(this.instruments);}
}
Interestingly, I am not maintaining the structure of the original file. Writing XM files isn't something I care about. To slightly repeat a point, there are many ways to create & edit XM files. I'm not building a project to do that. Assuming the two lists aren't re-sorted, it is not difficult to recreate the original file if that ever becomes important.
These next few classes are going to look very similar. They do a few things that would make Java purists angry. I consider that a benefit. Things like having a private member that's a string but is set by passing an array of bytes. There is no set method that accepts a string. The get returns a string and not a byte array (see previous statement about not building an XM writer). This is "built for purpose" code, the best kind of code.
Alright, so first is the header because it's first:
public class XMHeader{
/*
* Module ID
*/
private String id;
public void setId(byte[] b){this.id=(new String(b,StandardCharsets.UTF_8));}
public String getId(){return(this.id);}
/*
* Module name
*/
private String name;
public void setName(byte[] b){this.name=(new String(b,StandardCharsets.UTF_8));}
public String getName(){return(this.name);}
/*
* Tracker name
*/
private String trackerName;
public void setTrackerName(byte[] b){this.trackerName=(new String(b,StandardCharsets.UTF_8));}
public String getTrackerName(){return(this.trackerName);}
/*
* Version number, which is really a string here
*/
private String versionNumber;
public void setVersionNumber(byte[] b){
this.versionNumber=new String();
//should only be 2 bytes though
for(byte y:b){
String s=Byte.toString(y);
if(s.length()<2){
this.versionNumber="0"+s+this.versionNumber;
}else{
this.versionNumber=s+this.versionNumber;
}
}
}
public String getVersionNumber(){return(this.versionNumber);}
/*
* Header size
*/
private int headerSize;
public void setHeaderSize(byte[] b){this.headerSize=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getHeaderSize(){return(this.headerSize);}
/*
* Song length
*/
private int songLength;
public void setSongLength(byte[] b){this.songLength=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getSongLength(){return(this.songLength);}
/*
* Restart position
*/
private int restartPosition;
public void setRestartPosition(byte[] b){this.restartPosition=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getRestartPosition(){return(this.restartPosition);}
/*
* Number of channels
*/
private int numChannels;
public void setNumChannels(byte[] b){this.numChannels=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getNumChannels(){return(this.numChannels);}
/*
* Number of patterns
*/
private int numPatterns;
public void setNumPatterns(byte[] b){this.numPatterns=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getNumPatterns(){return(this.numPatterns);}
/*
* Number of instruments
*/
private int numInstruments;
public void setNumInstruments(byte[] b){this.numInstruments=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getNumInstruments(){return(this.numInstruments);}
/*
* Flags
*/
private int flags;
public void setFlags(byte[] b){this.flags=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getFlags(){return(this.flags);}
/*
* Default tempo
*/
private int defaultTempo;
public void setDefaultTempo(byte[] b){this.defaultTempo=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getDefaultTempo(){return(this.defaultTempo);}
/*
* Default BPM
*/
private int defaultBPM;
public void setDefaultBPM(byte[] b){this.defaultBPM=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getDefaultBPM(){return(this.defaultBPM);}
/*
* Pattern order table
*/
private byte[] patternOrderTable;
public void setPatternOrderTable(byte[] b){this.patternOrderTable=Arrays.copyOf(b,b.length);}
public byte[] getPatternOrderTable(){return(this.patternOrderTable);}
}
I suppose if there is room for improvement, depending on your definition of "improvement", it would be validating data length. This is hinted at in setVersionNumber. All these fields are fixed length and those rules are only enforced in the reader class a bit further down. Each set method could validate the length of the byte array against the corresponding constant and throw an exception if there is a mismatch.
This is a common design discussion. Should a class used to represent a data structure perform any validation or is it the responsibility of the class loading the data or both? If this was going into an internet-facing application you'd want to validate literally everywhere you can to prevent injection attacks. For a class that might eventually be used to generate music for Sega Genesis demos this is not a concern.
The next class is for patterns, which are a simple structure:
public class XMPattern{
/*
* Header size
*/
private int headerSize;
public void setHeaderSize(byte[] b){this.headerSize=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getHeaderSize(){return(this.headerSize);}
/*
* Number of rows
*/
private int numRows;
public void setNumRows(byte[] b){this.numRows=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getNumRows(){return(this.numRows);}
/*
* Pattern data size
*/
private int patternDataSize;
public void setPatternDataSize(byte[] b){this.patternDataSize=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getPatternDataSize(){return(this.patternDataSize);}
/*
* Pattern data
*/
private byte[] patternData;
public void setPatternData(byte[] b){this.patternData=Arrays.copyOf(b,b.length);}
public byte[] getPatternData(){return(this.patternData);}
}
Instruments is where things start to get complicated. In the XM specification, samples are contained within instruments. Maybe there's some argument they could be decoupled but that's what the reality is and we can't change it. Instruments may or may not have samples though. They have four fields (size, name, type, number of samples) always, and other fields that are optional depending on whether there are samples.
These rules are not enforced in the data class, they are enforced in reader again. So the base instrument class is light:
public class XMInstrument{
/*
* Header size
*/
private int headerSize;
public void setHeaderSize(byte[] b){this.headerSize=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getHeaderSize(){return(this.headerSize);}
/*
* Instrument name
*/
private String name;
public void setName(byte[] b){this.name=(new String(b,StandardCharsets.UTF_8));}
public String getName(){return(this.name);}
/*
* Instrument type
*/
private byte type;
public void setType(byte b){this.type=b;}
public byte getType(){return(this.type);}
/*
* Number of samples
*/
private int numSamples;
public void setNumSamples(byte[] b){this.numSamples=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getNumSamples(){return(this.numSamples);}
/*
* Instrument headers
*/
private XMInstrumentHeader instrumentHeader;
public void setInstrumentHeader(XMInstrumentHeader instrumentHeader){this.instrumentHeader=instrumentHeader;}
public XMInstrumentHeader getInstrumentHeader(){return(this.instrumentHeader);}
/*
* Sample headers
*/
private List<XMSampleHeader> sampleHeaders;
public void setSampleHeaders(List<XMSampleHeader> sampleHeaders){this.sampleHeaders=sampleHeaders;}
public List<XMSampleHeader> getSampleHeaders(){return(this.sampleHeaders);}
/*
* Sample data
*/
private List<XMSampleData> sampleData;
public void setSampleData(List<XMSampleData> sampleData){this.sampleData=sampleData;}
public List<XMSampleData> getSampleData(){return(this.sampleData);}
}
The optional/extra instrument header is a long class but also simple. Many fields are just single bytes which leads to the perpetual discussion about whether to use the primitive byte class or the object class. I went with the primitive class, changing it would be a very small effort if there is ever a reason to.
public class XMInstrumentHeader{
/*
* Header size
*/
private int headerSize;
public void setHeaderSize(byte[] b){this.headerSize=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getHeaderSize(){return(this.headerSize);}
/*
* "Sample number for all notes" or "sample keymap assignments" in other documentation
*/
private byte[] noteSampleNumbers;
public void setNoteSampleNumbers(byte[] b){this.noteSampleNumbers=Arrays.copyOf(b,b.length);}
public byte[] getNoteSampleNumbers(){return(this.noteSampleNumbers);}
/*
* Points for volume envelope
*/
private byte[] envelopeVolumePoints;
public void setEnvelopeVolumePoints(byte[] b){this.envelopeVolumePoints=Arrays.copyOf(b,b.length);}
public byte[] getEnvelopeVolumePoints(){return(this.envelopeVolumePoints);}
/*
* Points for panning envelope
*/
private byte[] envelopePanningPoints;
public void setEnvelopePanningPoints(byte[] b){this.envelopePanningPoints=Arrays.copyOf(b,b.length);}
public byte[] getEnvelopePanningPoints(){return(this.envelopePanningPoints);}
/*
* Number of volume points
*/
private byte numVolumePoints;
public void setNumVolumePoints(byte b){this.numVolumePoints=b;}
public byte getNumVolumePoints(){return(this.numVolumePoints);}
/*
* Number of panning points
*/
private byte numPanningPoints;
public void setNumPanningPoints(byte b){this.numPanningPoints=b;}
public byte getNumPanningPoints(){return(this.numPanningPoints);}
/*
* Volume sustain point
*/
private byte volumeSustainPoint;
public void setVolumeSustainPoint(byte b){this.volumeSustainPoint=b;}
public byte getVolumeSustainPoint(){return(this.volumeSustainPoint);}
/*
* Volume loop start point
*/
private byte volumeLoopStartPoint;
public void setVolumeLoopStartPoint(byte b){this.volumeLoopStartPoint=b;}
public byte getVolumeLoopStartPoint(){return(this.volumeLoopStartPoint);}
/*
* Volume loop end point
*/
private byte volumeLoopEndPoint;
public void setVolumeLoopEndPoint(byte b){this.volumeLoopEndPoint=b;}
public byte getVolumeLoopEndPoint(){return(this.volumeLoopEndPoint);}
/*
* Panning sustain point
*/
private byte panningSustainPoint;
public void setPanningSustainPoint(byte b){this.panningSustainPoint=b;}
public byte getPanningSustainPoint(){return(this.panningSustainPoint);}
/*
* Panning loop start point
*/
private byte panningLoopStartPoint;
public void setPanningLoopStartPoint(byte b){this.panningLoopStartPoint=b;}
public byte getPanningLoopStartPoint(){return(this.panningLoopStartPoint);}
/*
* Panning loop end point
*/
private byte panningLoopEndPoint;
public void setPanningLoopEndPoint(byte b){this.panningLoopEndPoint=b;}
public byte getPanningLoopEndPoint(){return(this.panningLoopEndPoint);}
/*
* Volume type
*/
private byte volumeType;
public void setVolumeType(byte b){this.volumeType=b;}
public byte getVolumeType(){return(this.volumeType);}
/*
* Panning type
*/
private byte panningType;
public void setPanningType(byte b){this.panningType=b;}
public byte getPanningType(){return(this.panningType);}
/*
* Vibrato type
*/
private byte vibratoType;
public void setVibratoType(byte b){this.vibratoType=b;}
public byte getVibratoType(){return(this.vibratoType);}
/*
* Vibrato sweep
*/
private byte vibratoSweep;
public void setVibratoSweep(byte b){this.vibratoSweep=b;}
public byte getVibratoSweep(){return(this.vibratoSweep);}
/*
* Vibrato depth
*/
private byte vibratoDepth;
public void setVibratoDepth(byte b){this.vibratoDepth=b;}
public byte getVibratoDepth(){return(this.vibratoDepth);}
/*
* Vibrato rate
*/
private byte vibratoRate;
public void setVibratoRate(byte b){this.vibratoRate=b;}
public byte getVibratoRate(){return(this.vibratoRate);}
/*
* Volume fadeout
*/
private int volumeFadeout;
public void setVolumeFadeout(byte[] b){this.volumeFadeout=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getVolumeFadeout(){return(this.volumeFadeout);}
/*
* Reserved block
*/
private byte[] reserved;
public void setReserved(byte[] b){this.reserved=Arrays.copyOf(b,b.length);}
public byte[] getReserved(){return(this.reserved);}
}
Yeah, these all look very much the same after a while. Here's the sample header data class:
public class XMSampleHeader{
/*
* Sample length
*/
private int sampleLength;
public void setSampleLength(byte[] b){this.sampleLength=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getSampleLength(){return(this.sampleLength);}
/*
* Loop start
*/
private int sampleLoopStart;
public void setSampleLoopStart(byte[] b){this.sampleLoopStart=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getSampleLoopStart(){return(this.sampleLoopStart);}
/*
* Loop length
*/
private int sampleLoopLength;
public void setSampleLoopLength(byte[] b){this.sampleLoopLength=NumberFormatters.byteArrayToInt(b,Endianness.LITTLE_ENDIAN);}
public int getSampleLoopLength(){return(this.sampleLoopLength);}
/*
* Volume
*/
private byte volume;
public void setVolume(byte b){this.volume=b;}
public byte getVolume(){return(this.volume);}
/*
* Finetune
*/
private byte finetune;
public void setFinetune(byte b){this.finetune=b;}
public byte getFinetune(){return(this.finetune);}
/*
* Type
*/
private byte type;
public void setType(byte b){this.type=b;}
public byte getType(){return(this.type);}
/*
* Panning
*/
private byte panning;
public void setPanning(byte b){this.panning=b;}
public byte getPanning(){return(this.panning);}
/*
* Relative note number
*/
private byte relativeNoteNumber;
public void setRelativeNoteNumber(byte b){this.relativeNoteNumber=b;}
public byte getRelativeNoteNumber(){return(this.relativeNoteNumber);}
/*
* Packing
*/
private byte packing;
public void setPacking(byte b){this.packing=b;}
public byte getPacking(){return(this.packing);}
/*
* Name
*/
private String name;
public void setName(byte[] b){this.name=(new String(b,StandardCharsets.UTF_8));}
public String getName(){return(this.name);}
}
Sample data I'm storing raw. It is not used for the existing XM to Echo conversion application, I may one day take a stab at a sample data convertor though.
public class XMSampleData{
private byte[] rawData;
public void setRawData(byte[] b){this.rawData=Arrays.copyOf(b,b.length);}
public byte[] getRawData(){return(this.rawData);}
//constructor that accepts the raw data
public XMSampleData(byte[] b){super(); this.setRawData(b);}
}
All the field lengths and other rules are enforced in the reader. Let's break it down into a few separate parts.
It starts with simply reading the entire XM file into memory. This is another design trade-off discussion - read the whole file or stream it? The largest XM file I have is under 8mb so I'm just loading the whole thing. Again, in a different system it may be preferable to stream the file. Like if you needed to read some absurd number in parallel and had limited memory. Hmm, if you had limited memory though then the overhead of thread management might be an issue before the memory needed for the files themselves. Anyway...
public static XMFile read(File f) throws Exception{
XMFile xm=new XMFile();
/*
* This assumes most xm files are small and reading the entire file won't be a memory issue.
* I don't know why this would ever be used in a way this is an issue.
*
* This is also written to make debugging easier.
* Yes, this could be written in fewer lines if that is a thing important to anyone.
*/
byte[] b=Files.readAllBytes(f.toPath());
XMHeader header=new XMHeader();
The next part is boring, reading the XM header:
/*
* Header -> Module ID
*/
int start=0;
int end=start+XMConstants.FieldLengths.HEADER_ID;
header.setId(Arrays.copyOfRange(b,start,end));
/*
* Header -> Module name
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_MODULENAME;
header.setName(Arrays.copyOfRange(b,start,end));
/*
* Header -> Tracker name
*/
start=end+1;//+1 for fixed field after modulename
end=start+XMConstants.FieldLengths.HEADER_TRACKERNAME;
header.setTrackerName(Arrays.copyOfRange(b,start,end));
/*
* Header -> Version number
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_VERSION;
header.setVersionNumber(Arrays.copyOfRange(b,start,end));
/*
* Header -> Header size
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SIZE;
header.setHeaderSize(Arrays.copyOfRange(b,start,end));
/*
* Header -> Song length
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SONG_LENGTH;
header.setSongLength(Arrays.copyOfRange(b,start,end));
/*
* Header -> Song restart position
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SONG_RESTART;
header.setRestartPosition(Arrays.copyOfRange(b,start,end));
/*
* Header -> Song number of channels
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SONG_NUM_CHANNELS;
header.setNumChannels(Arrays.copyOfRange(b,start,end));
/*
* Header -> Song number of patterns
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SONG_NUM_PATTERNS;
header.setNumPatterns(Arrays.copyOfRange(b,start,end));
/*
* Header -> Song number of instruments
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SONG_NUM_INSTRUMENTS;
header.setNumInstruments(Arrays.copyOfRange(b,start,end));
/*
* Header -> Flags
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SONG_FLAGS;
header.setFlags(Arrays.copyOfRange(b,start,end));
/*
* Header -> Default tempo
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SONG_TEMPO;
header.setDefaultTempo(Arrays.copyOfRange(b,start,end));
/*
* Header -> Default BPM
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SONG_BPM;
header.setDefaultBPM(Arrays.copyOfRange(b,start,end));
/*
* Header -> Pattern order table
*/
start=end;
end=start+XMConstants.FieldLengths.HEADER_SONG_PATTERN_ORDER;
header.setPatternOrderTable(Arrays.copyOfRange(b,start,end));
xm.setHeader(header);
Patterns are slightly less boring because we get to make a list. As noted earlier, sorting that list would be a bad idea unless the pattern order table is also sorted to match.
/*
* Now patterns
*/
int patternStart=end;
ArrayList<XMPattern> patterns=new ArrayList<XMPattern>();
int patternCount=header.getNumPatterns();
for(int patternIndex=0;patternIndex<patternCount;patternIndex++){
XMPattern pattern=new XMPattern();
/*
* Pattern -> header length
*/
start=patternStart;
end=start+FieldLengths.PATTERN_HEADER;
pattern.setHeaderSize(Arrays.copyOfRange(b,start,end));
/*
* Pattern -> number of rows
*/
start=patternStart+XMConstants.Offsets.PATTERN_NUM_ROWS;
end=start+FieldLengths.PATTERN_NUM_ROWS;
pattern.setNumRows(Arrays.copyOfRange(b,start,end));
/*
* Pattern -> data size
*/
start=patternStart+XMConstants.Offsets.PATTERN_DATA_SIZE;
end=start+FieldLengths.PATTERN_DATA_SIZE;
pattern.setPatternDataSize(Arrays.copyOfRange(b,start,end));
/*
* Pattern -> data
*/
int patternDataSize=pattern.getPatternDataSize();
start=end;
end=start+patternDataSize;
pattern.setPatternData(Arrays.copyOfRange(b,start,end));
patterns.add(pattern);
patternStart=end;
}
xm.setPatterns(patterns);
The first part of reading instruments is easy enough:
/*
* Now instruments
*/
ArrayList<XMInstrument> instruments=new ArrayList<XMInstrument>();
int numInstruments=xm.getHeader().getNumInstruments();
for(int instrumentNumber=0;instrumentNumber<numInstruments;instrumentNumber++){
XMInstrument instrument=new XMInstrument();
/*
* Instrument -> header size
*/
start=end;
end=start+XMConstants.FieldLengths.INSTRUMENT_HEADER_SIZE;
instrument.setHeaderSize(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> name
*/
start=end;
end=start+XMConstants.FieldLengths.INSTRUMENT_NAME;
instrument.setName(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> type
*/
start=end;
end=start+XMConstants.FieldLengths.INSTRUMENT_TYPE;
instrument.setType(b[start]);
/*
* Instrument -> number of samples
*/
start=end;
end=start+XMConstants.FieldLengths.INSTRUMENT_NUM_SAMPLES;
instrument.setNumSamples(Arrays.copyOfRange(b,start,end));
instruments.add(instrument);
int numSamples=instrument.getNumSamples();
Still easy but... if there are samples for the instrument we need to read the additional header:
if(numSamples>0){
XMInstrumentHeader instrumentHeader=new XMInstrumentHeader();
/*
* Instrument -> header -> header size
*/
start=end;
end=start+XMConstants.FieldLengths.INSTRUMENT_HEADER_SIZE;
instrumentHeader.setHeaderSize(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> header -> keymap
*/
start=end;
end=start+XMConstants.FieldLengths.INSTRUMENT_SAMPLENUMBER_SIZE;
instrumentHeader.setNoteSampleNumbers(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> header -> volume envelope
*/
start=end;
end=start+XMConstants.FieldLengths.INSTRUMENT_ENVELOPEPOINTS_SIZE;
instrumentHeader.setEnvelopeVolumePoints(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> header -> panning envelope
*/
start=end;
end=start+XMConstants.FieldLengths.INSTRUMENT_ENVELOPEPOINTS_SIZE;
instrumentHeader.setEnvelopePanningPoints(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> header -> number of volume points
*/
start=end;
instrumentHeader.setNumVolumePoints(b[start]);
/*
* Instrument -> header -> number of panning points
*/
start++;
instrumentHeader.setNumPanningPoints(b[start]);
/*
* Instrument -> header -> volume sustain point
*/
start++;
instrumentHeader.setVolumeSustainPoint(b[start]);
/*
* Instrument -> header -> volume loop start point
*/
start++;
instrumentHeader.setVolumeLoopStartPoint(b[start]);
/*
* Instrument -> header -> volume loop end point
*/
start++;
instrumentHeader.setVolumeLoopEndPoint(b[start]);
/*
* Instrument -> header -> panning sustain point
*/
start++;
instrumentHeader.setPanningSustainPoint(b[start]);
/*
* Instrument -> header -> panning loop start point
*/
start++;
instrumentHeader.setPanningLoopStartPoint(b[start]);
/*
* Instrument -> header -> panning loop end point
*/
start++;
instrumentHeader.setPanningLoopEndPoint(b[start]);
/*
* Instrument -> header -> volume type
*/
start++;
instrumentHeader.setVolumeType(b[start]);
/*
* Instrument -> header -> panning type
*/
start++;
instrumentHeader.setPanningType(b[start]);
/*
* Instrument -> header -> vibrato type
*/
start++;
instrumentHeader.setVibratoType(b[start]);
/*
* Instrument -> header -> vibrato sweep
*/
start++;
instrumentHeader.setVibratoSweep(b[start]);
/*
* Instrument -> header -> vibrato depth
*/
start++;
instrumentHeader.setVibratoDepth(b[start]);
/*
* Instrument -> header -> vibrato rate
*/
start++;
instrumentHeader.setVibratoRate(b[start]);
/*
* Instrument -> header -> volume fadeout
*/
start++;
end=start+XMConstants.FieldLengths.INSTRUMENT_VOLUMEFADEOUT_SIZE;
instrumentHeader.setVolumeFadeout(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> header -> reserved
*/
start=end;
end=start+XMConstants.FieldLengths.INSTRUMENT_RESERVED_SIZE;
instrumentHeader.setReserved(Arrays.copyOfRange(b,start,end));
instrument.setInstrumentHeader(instrumentHeader);
After that all the sample headers are read because samples are part of the instrument object.
/*
* Sample headers
*/
ArrayList<XMSampleHeader> sampleHeaders=new ArrayList<XMSampleHeader>();
for(int i=0;i<numSamples;i++){
XMSampleHeader sampleHeader=new XMSampleHeader();
/*
* Instrument -> sample header -> sample length
*/
start=end;
end=start+XMConstants.FieldLengths.SAMPLE_HEADER_LENGTH;
sampleHeader.setSampleLength(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> sample header -> Loop start
*/
start=end;
end=start+XMConstants.FieldLengths.SAMPLE_HEADER_LOOPSTART;
sampleHeader.setSampleLoopStart(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> sample header -> Loop length
*/
start=end;
end=start+XMConstants.FieldLengths.SAMPLE_HEADER_LOOPLENGTH;
sampleHeader.setSampleLoopLength(Arrays.copyOfRange(b,start,end));
/*
* Instrument -> sample header -> Volume
*/
start=end;
sampleHeader.setVolume(b[start]);
/*
* Instrument -> sample header -> Finetune
*/
start++;
sampleHeader.setFinetune(b[start]);
/*
* Instrument -> sample header -> Type
*/
start++;
sampleHeader.setType(b[start]);
/*
* Instrument -> sample header -> Panning
*/
start++;
sampleHeader.setPanning(b[start]);
/*
* Instrument -> sample header -> Relative note number
*/
start++;
sampleHeader.setRelativeNoteNumber(b[start]);
/*
* Instrument -> sample header -> Packing
*/
start++;
sampleHeader.setPacking(b[start]);
/*
* Instrument -> sample header -> Name
*/
start++;
end=start+XMConstants.FieldLengths.SAMPLE_HEADER_NAME;
sampleHeader.setName(Arrays.copyOfRange(b,start,end));
sampleHeaders.add(sampleHeader);
}
instrument.setSampleHeaders(sampleHeaders);
The last part read is the sample data and then we loop to the next instrument:
/*
* Read sample data
*/
ArrayList<XMSampleData> sampleData=new ArrayList<XMSampleData>();
for(int i=0;i<numSamples;i++){
int sampleLength=instrument.getSampleHeaders().get(i).getSampleLength();
start=end;
end=start+sampleLength;
sampleData.add(new XMSampleData(Arrays.copyOfRange(b,start,end)));
}
instrument.setSampleData(sampleData);
}//if(numSamples>0)
}//for instrument loop
xm.setInstruments(instruments);
return(xm);
}
}
To validate this, I wrote a quick tester that loads an XM file and then dumps all the fields to the console. I didn't add this to Github and it's not really noteworthy code. It's exactly what I just described if you want to visualize it.
I used the XM file that was also used for the Retail Clerk '89 title screen for testing. Here's the abridged output from the tester:
xm.getHeader().getId(): Extended Module:
xm.getHeader().getName(): CHiP_OVERTURE
xm.getHeader().getTrackerName(): MilkyTracker
xm.getHeader().getVersionNumber(): 0104
xm.getHeader().getHeaderSize(): 276
xm.getHeader().getSongLength(): 23
xm.getHeader().getRestartPosition(): 0
xm.getHeader().getNumChannels(): 10
xm.getHeader().getNumPatterns(): 20
xm.getHeader().getNumInstruments(): 12
xm.getHeader().getFlags(): 1
xm.getHeader().getDefaultTempo(): 6
xm.getHeader().getDefaultBPM(): 160
xm.getHeader().getPatternOrderTable().length: 256
xm.getHeader().getPatternOrderTable():
2 3 4 5 6 7 7 8 9 0 0 10 11 12 13 14 15 16 17 16 17 18 19 [rest of table clipped]
--end of header--
pattern[0].getHeaderSize(): 9
pattern[0].getNumRows(): 64
pattern[0].getPatternDataSize(): 1694
pattern[0].getPatternData(): [clipped]
[... and so on ...]
pattern[19].getHeaderSize(): 9
pattern[19].getNumRows(): 64
pattern[19].getPatternDataSize(): 1496
pattern[19].getPatternData(): [clipped]
---end of patterns--
instrument[0].getHeaderSize(): 263
instrument[0].getName(): CHiP_OVERTURE
instrument[0].getType(): 0
instrument[0].getNumSamples(): 1
instrument[0].instrumentHeader.getHeaderSize(): 40
instrument[0].instrumentHeader.getNoteSampleNumbers().length: 96
instrument[0].instrumentHeader.getNoteSampleNumbers(): [clipped - all 0s]
instrument[0].instrumentHeader.getEnvelopeVolumePoints().length: 48
instrument[0].instrumentHeader.getEnvelopeVolumePoints(): [clipped]
instrument[0].instrumentHeader.getEnvelopePanningPoints().length: 48
instrument[0].instrumentHeader.getEnvelopePanningPoints(): [clipped]
instrument[0].instrumentHeader.getNumVolumePoints(): 4
instrument[0].instrumentHeader.getNumPanningPoints(): 6
instrument[0].instrumentHeader.getVolumeSustainPoint(): 1
instrument[0].instrumentHeader.getVolumeLoopStartPoint(): 1
instrument[0].instrumentHeader.getVolumeLoopEndPoint(): 3
instrument[0].instrumentHeader.getPanningSustainPoint(): 2
instrument[0].instrumentHeader.getPanningLoopStartPoint(): 3
instrument[0].instrumentHeader.getPanningLoopEndPoint(): 5
instrument[0].instrumentHeader.getVolumeType(): 5
instrument[0].instrumentHeader.getPanningType(): 0
instrument[0].instrumentHeader.getVibratoType(): 0
instrument[0].instrumentHeader.getVibratoSweep(): 0
instrument[0].instrumentHeader.getVibratoDepth(): 0
instrument[0].instrumentHeader.getVibratoRate(): 0
instrument[0].instrumentHeader.getVolumeFadeout(): 1536
instrument[0].instrumentHeader.getReserved().length: 22
instrument[0].instrumentHeader.getReserved(): [clipped]
--end of instrument header--
instrument[0].sampleHeaders.size(): 1
instrument[0].sampleHeader[0].getSampleLength(): 200
instrument[0].sampleHeader[0].getSampleLoopStart(): 0
instrument[0].sampleHeader[0].getSampleLoopLength(): 200
instrument[0].sampleHeader[0].getVolume(): 64
instrument[0].sampleHeader[0].getFinetune(): -4
instrument[0].sampleHeader[0].getType(): 1
instrument[0].sampleHeader[0].getPanning(): 0
instrument[0].sampleHeader[0].getRelativeNoteNumber(): 32
instrument[0].sampleHeader[0].getPacking(): 0
instrument[0].sampleHeader[0].getName():
--end of sample headers--
instrument[0].getSampleData().length: 200
instrument[0].instrument.getSampleData(): [clipped]
--end of sample data--
[..and so on...]
instrument[11].getHeaderSize(): 263
instrument[11].getName():
instrument[11].getType(): 0
instrument[11].getNumSamples(): 1
instrument[11].instrumentHeader.getHeaderSize(): 40
instrument[11].instrumentHeader.getNoteSampleNumbers().length: 96
instrument[11].instrumentHeader.getNoteSampleNumbers(): [clipped]
instrument[11].instrumentHeader.getEnvelopeVolumePoints().length: 48
instrument[11].instrumentHeader.getEnvelopeVolumePoints(): [clipped]
instrument[11].instrumentHeader.getEnvelopePanningPoints().length: 48
instrument[11].instrumentHeader.getEnvelopePanningPoints(): [clipped]
instrument[11].instrumentHeader.getNumVolumePoints(): 6
instrument[11].instrumentHeader.getNumPanningPoints(): 6
instrument[11].instrumentHeader.getVolumeSustainPoint(): 2
instrument[11].instrumentHeader.getVolumeLoopStartPoint(): 3
instrument[11].instrumentHeader.getVolumeLoopEndPoint(): 5
instrument[11].instrumentHeader.getPanningSustainPoint(): 2
instrument[11].instrumentHeader.getPanningLoopStartPoint(): 3
instrument[11].instrumentHeader.getPanningLoopEndPoint(): 5
instrument[11].instrumentHeader.getVolumeType(): 7
instrument[11].instrumentHeader.getPanningType(): 0
instrument[11].instrumentHeader.getVibratoType(): 0
instrument[11].instrumentHeader.getVibratoSweep(): 0
instrument[11].instrumentHeader.getVibratoDepth(): 0
instrument[11].instrumentHeader.getVibratoRate(): 0
instrument[11].instrumentHeader.getVolumeFadeout(): 128
instrument[11].instrumentHeader.getReserved().length: 22
instrument[11].instrumentHeader.getReserved(): [clipped]
--end of instrument header--
instrument[11].sampleHeaders.size(): 1
instrument[11].sampleHeader[0].getSampleLength(): 3000
instrument[11].sampleHeader[0].getSampleLoopStart(): 0
instrument[11].sampleHeader[0].getSampleLoopLength(): 3000
instrument[11].sampleHeader[0].getVolume(): 64
instrument[11].sampleHeader[0].getFinetune(): 0
instrument[11].sampleHeader[0].getType(): 2
instrument[11].sampleHeader[0].getPanning(): -128
instrument[11].sampleHeader[0].getRelativeNoteNumber(): 12
instrument[11].sampleHeader[0].getPacking(): 0
instrument[11].sampleHeader[0].getName():
--end of sample headers--
instrument[11].getSampleData().length: 3000
instrument[11].instrument.getSampleData(): [clipped]
--end of sample data--
--end of instruments--
OK, so this worked with one specific file that I already knew could be converted to Echo format. I suspect it would work with any valid XM file but also know from experience it's laughable to actually believe that. If/when I try more files I will find some undocumented nuance in 1% of files that all other XM readers already accounted for. That I'm very confident of. Those changes will be here when I run into them.
Related
