Hexagonal architecture abstractions for file system interactions according to xdg basedir specs

When you are going to design architectures like Hexagonal or some other "Clean Architecture" variant, it works best when you don’t abstract "files", but features of your domain.

In other words, your domain shouldn’t care about std::fs, std::path, FILE*, or XDG rules. It cares about things like:

• persist a stream of bytes under key X

• load a configuration

• append an event log

• store a cache artifact

When you phrase this way, the abstraction becomes much smaller and you don’t need to re-implement a full filesystem.

I am not a Rust expert, so let me scetch an example in C++ terms. Let's say you implement your ports by using abstract classes, and adapters by implementing those classes. Then you don't design an abstract class encapsulating all kind of individual file system operations. Instead, ask yourself what kind of state you want to persist, in domain terms? Lets say the state are configuration options, and design the "port" around this:

 struct ConfigurationStorage
 {
     // may throw an XYZException when saving fails
     virtual void Save(const Configuration &)=0;
     // returns a new default configuration when there is no
     // former config stored;
     // may throw an XYZException when loading fails for other reasons
     virtual Configuration Load()=0;
 }

The adapter will be the implementation of this class, hiding all file system operations as well as all access to the XDG environment variables away from the core domain / core application.

Of course, in reality you may need some more methods for this class, maybe you want to control certain file name parts, or let the adapter create certain subdirectories automatically. But as long as you focus on domain terms and domain abstractions, these ports stay far more high level as the whole file system API.

Hexagonal Architecture has many diagrams at this point. Here's one that explicitly puts the file system outside:

itnext.io - Hexagonal Architecture Principles Practical Example In Java

However, most Hex diagrams don't mention the file system at all. Instead they talk about a need in an abstract way. They do this because it doesn't mater to the architecture what satisfies that need. Here's one that talks about the need for persistence.

reflectoring.io - Spring Hexagonal

That persistence may be a file system. Or a Data Base. Or a chisel and stone tablet. The architecture doesn't care. It put it out here and called it that so it wouldn't care.

Now this doesn't say all file system use must be regulated out here. It says persistence concerns are regulated out here. So if you find some other use for the file system the first question should be, what is the real abstract need for this use? And, regardless of the decision to solve it with the file system or something else, where should solutions to that need live?

Thinking like that will keep your architecture organized and flexible.

"I want to persist state in directories conforming to xdg base dir specs". Now what architectural design decisions can i derive from that? - Samuel Beer

None. Unless your decision is I want to force everything to do it this way. In which case you spread that decision into every module and write all the code you can assuming that decision will always hold.

If that's not the choice you want to make then these diagrams are telling you to contain all code that cares about that decision to this one little module and let the rest of the code not care.

Do that and changing your mind later is much less work.

So what would I persist in the ideal world? Samuel Beer

Concepts from your domain. If your app is about internet cat videos then those videos, and whatever associated meta data, become what you must find some way to persist. It's nice if the persistence module is the only part of your app that knows you decided to use xdg-basedir specs. By isolating that decision/implementation detail you make it easy to change later. The high art here, you still need to find some way (a protocol, an interface, an output port) for the rest of the app to talk to your persistence module. But, since we abstracted away the xdg-basedir details we're free to design that persistence protocol thinking only about the domains real needs (which would be access to cat videos). This protocol/interface is what will get spread around instead. So it's important to get it right. Because it needs to be stable. Because once it has been spread around it wont be easy to change. Which is why volatile details, like you chose to use the file system, are not welcome here. So we use the adapter pattern to hide that detail from the rest of the app.

Notice that this is all about diving from the higher level concept of 'hexagonal architecture' into solution space which I find I have a hard time to do. Samuel Beer

That can be a challenge. What guides me is needs. What does the rest of the app need once the persistence problem is solved? It needs the video. One abstraction that might be useful here is a stream. Choosing a stream doesn't force you to use a file system. It's a useful and popular abstraction. But it's not the only way to do this. It's the decisions you can't help spreading around that you need to be the most careful with because they're the hardest to change.

Methinks you're mixing layers and get confused by that. Hexagonal is (IMO) way above the details of accessing files and directories, is there any reason you want to encumber filesystem interaction with it?

For file access, using std::fs (or the tokio equivalent) should be fine. That covers your bullet points 1, 2, possibly 3 (using std::fs:create_all()) , and 5. For 3, you might need to do the right thing with access rights after creating the parent directories.

For point 4, XDG base specs may be easiest handled with a library that provides the necessary directory locations and search functions based on the process environment. Apparently such a library already exists (https://docs.rs/xdg/latest/xdg/), is there any reason not to use that?

I don't think I get where you see the problem with testing - if you wrote some bytes to a file in whose name was determined using the xdg library, it should be easy to read that file again and check its contents.