You should definitely benchmark it using a simulation of what your data might look like. It should not be that hard. Just make script, that creates bunch of files similar to your data.

Right, it’s just kind of a thing to think about. If your program is something that might conceivably be used of sshfs (as an example) … this is probably not a great option for your program’s configuration.

The data structure and parser is not formed the same way as the json, where you have to parse the whole thing. So you don’t have to. You just open the files you need read their content. It is a bit more difficult at first since you can’t just translate a whole struct directly, but it pays for itself when you want to migrate the data to a new format. So if your structure never changes, probably those formats are easier.

Well a very common thing is to create a “config” object that lives in the long running process (and in some cases can be reloaded without restarting the program).

That model also saves you from unnecessary repeated IO operations (without one off caching and reloading mechanisms) and allows you to centralize any validation (which also means you can give configuration errors on start up).

I do wish various formats were more “streaming” friendly, but configuration isn’t really one of them.

In a lot of languages moving between formats is also fairly trivial because the XYZ markup parser parses things into an object map and the ZYK markup writer can write an object map into ZYK format.

Maybe I’m not understanding what you mean by migrating the data to a new format though.