People have many different opinions about config systems. Having worked with various styles of configs, I also want to write about what a great config subsystem in a large-scale (in terms of system complexity, number of users, etc.) system should look like.

The design space is complex, so in this article I'll start with a smaller topic: registration in config systems. I'll show why this common pattern, though works fine for small-scale projects, does not scale well in the long term. I'll also discuss an alternative.

"Loss function" is one of the most basic concepts today in deep learning. Despite that, it is actually not necessarily a good programming abstraction when designing general-purpose systems. A system should not assume that a model always comes together with a separate "loss function".

Building a library for research and experiments is quite different from building other types of software. A key challenge is that, in research, abstractions and APIs are rarely set in stone: users may want to propose a slight variant or modification to literally ANYWHERE in the whole program, just because they have a new idea.