I’m making a turn-based RPG and I’m in need of ways to store data. Thus far, I have been planning on using a series of XML docs to store save data, NPC data, dialogue, item info, monster info, and skill info. I’m also using Tiled maps, so I have those tmx files in my game as well. While I was researching XML usage, I realized that it’s on the slower side and that there are size limitations. The game is very narrative-heavy, so there will be a ton of dialogue ids. Also, I’m not using the Monogame content pipeline with the XMLs, if that’s relevant.
So, my question: are these XML docs I have planned too much? I already have a lot of work done on the XML side so I’d like to keep them if it’s fine, but I don’t want to run into issues as my game grows. I’ve been looking into sqlite and it seems like a good option for all the monster/npc/dialogue data, but I’d like to keep my save data as an XML-- is mixing data storage anti-best practice?
No problem mixing them. I use a combination of binary files and XML in one project and it works fine. XML works well for dialogue for me. In addition to XML and sqlite you can look at JSON. It’s fairly lightweight and there is good support for serialization.
That’s good to know! Did you run into any size limitations for XML dialogue storage? How much dialogue did that project have? I have working dialogue using XML, but I have no idea what will happen when I start writing a lot more
I have used over 100,000 objects in a list that serialize and deserialize to xml in a couple of seconds (probably a total of around 1,000,000 fields). This was on a hard drive, not an ssd.
I haven’t run into size issues with XML. I use it to store my maps too which tend up being more massive than the dialogue. Single map file is usually 10MB. Dialogue don’t get near that size in my experience.
WORD COUNTS OF THE BOOKS IN J.R.R. TOLKIEN’S LORD OF THE RINGS SERIES:
The Hobbit – 95,356 words
The Fellowship of the Ring – 187,790 words
The Two Towers – 156,198 words
The Return of the King – 137,115 words
The entire Lord of the Rings series (including The Hobbit) – 576,459 words
If we assume the average word contains say, five letters, just for the sake of this equation… and note that maths is nobody’s strong point… let us assume there are – taking 576,459 words multiplying by 5 to represent 5 characters per word, giving – 2,882,295 letters in the entire novel, we then assume we are using 8 bits per letter or (char) giving us 23,058,360 bits and if we convert that to Megabytes we get 2.7488 Megabytes according to this calculator site:
What do I mean by that, well let’s take a single boolean
The XML version of it will be something along the line of this
`<valuename>true</valuename>`
Roughly 29 bytes
The binary version will be a single byte, if you have a lot of booleans you could group them together and have a single bit for a boolean
Why is it insecure?
Well it’s human readable , so anyone with a text editor can read it and change it
Why is it slow to read?
You have to parse the structure to get the value, a binary version would be a single call to ReadByte()
Does this mean that XML is evil?
Yes and no, it depends on your game design. If you don’t care about the problems with XML, then the advantages of the structured human readable file are attractive.
For me personally, I never use XML anymore, if I need to I will use JSON in an editor (which is pretty much a sub set of XML I know), but never in game…
In game I only use binary data, I often structure data with L2 cache size in mind, so you can read every value in a structure without invalidating the cache, but then I am paranoid.
For me the best solution is to use something human readable in the editor, and binary in game.
One more question-- I haven’t been using the pipeline/mgcb editor, just the plain XML files and parsing with C#-- would that have longterm negative/slowing effects?
The reason for doing something like Sqlite vs xml is that you can store unstructured data in XML and treat it like a no-sql datastore.
Sqlite is a relational database, so for example you might have a Dungeon table, a Monster table, and a DungeonMonster table for storing instances of monsters that are contained in each dungeon. This table would only need a foreign key to the dungeon, a FK to the monster, and the location where the monster spawns in the dungeon.
If you are comfortable using XML, and aren’t used to thinking relationally anyway… I don’t see any reason why you should switch.
Maybe think about abstracting all the storage behind a repository pattern though so you can switch out your storage solutions easily. Personally, I usually do three different repositorys: an in-memory version for testing and quick development, a Sqlite version for simple local storage, and then the full-blown cloud implementation that integrates with PlayFab or Firebase.
If I may make a suggestion, try and fragment the information you need over multiple files. May them be json or XML, it should be fine as long as you break them off.
But consider the nature of the data you are using. I assume you have some dialogue data, items, triggers, areas, characters. How much of that you actually need in the memory at any one time?
If you break off your world into “areas”, then you should be fine. You’ll probably have different tilesets anyway for certain areas, so why not just do that?
Then, each area can load its own resources and you can manage them as you need, leaving the global resources at a minimum. By global assuming maybe your inventory/skills/stashes/heroes/quests progress and so on.
You could also try out the profiler in Visual Studio and monitor how much memory you are using at times, try and experiment with exaggerated lengths of data to push the limits and get some idea of minimum requirements for your game.
Nowadays, RAM is cheap, but HDD(SSD) is cheaper. If you can read and discard, do it.
Please do not get stuck on implementation details. For the sake of discussion, if it saves you a day of work, take this: “use xml and change later”.
That “later” I believe will come when your current implementation fails to satisfy the development requirements.
There’s a saying somewhere: no time like now. Just code, make something work. It’s easier to erase and rewind rather than not having recorded any kind of progress to learn something from.