I built a database. What should I do next?

Real systems are made over years and decades, by entire teams and divisions of engineers, in response to concrete pain points that someone needs solved. Nobody builds a skyscraper by himself.

Don't be discouraged by that. You have gained experience and you have learned something. See it as a journey because it is.

You decide your goals. Your goal could be to learn, or to build a database system for a specific need you have that nothing else serves. You could go to uni and study all of this, then go into research. Or you could apply to companies that build DBMS. Or you could look for open source DBMS projects, explore their code and issues.

You aren't a fool; you're reinventing the wheel. This is something we all do. Recreating something that was previously created allows you to explore the topic much deeper. Those analysis skills will take you a long way in software engineering.

This really did remind me: how could I possibly compare an undergraduate thesis project with the work of people who have spent more than a decade in the field? Anyone who does that is obviously a fool, and I am that fool.

It's impressive to have achieved what you did as an undergraduate.

You should not evaluate your project by comparing it to mature RDBMS projects that in fact have thousands of engineer-years of work behind them. Those projects have also benefited from customers using the RDBMS in many real production environments. Naturally, field experiences haved made those products more mature than yours.

So this is a learning project for you, and that's valuable to you. It's a toy in the sense that it's not being used for production work, but that's not a disparagement.

Thomas Jefferson said, "What we learn to do, we learn by doing."

You're getting practice facing the problems and fixing them one by one. You can't learn to paint, or play guitar, or speak in front of an audience — or any other professional skill — without practicing, making mistakes, and learning from the mistakes. Making the mistakes on a practice project is good, because the mistakes don't cause damage or financial loss, they only spend your time.

Spending time is a feature, not a defect, of the learning process. Solving problems is important to the learning process. It helps our brains retain the knowledge. That takes time.

If you're asking for ideas of what to add to your RDBMS learning project, I have a few. I've read about a few techniques, but never had the opportunity to implement them in an RDBMS. I've wondered if they would be useful features.

• Error detection and correction. I took a couple of information theory classes (the professor was David Huffman, who is known for Huffman Coding). By adding check bits to every information payload, you can create a system that detects errors, or with even more bits, correct errors. I've wondered if this could be used in an RDBMS to protect against data corruption. It would increase the storage requirement for data, of course, but for certain applications, a guarantee against data corruption could be important.

• Lockless algorithms. Multi-user RDBMS systems often experience a bottleneck because of locks, which are necessary to implement ACID. For example, if many threads are writing to the WAL, they have to use locks so only one thread at a time is writing. Otherwise the WAL records will become corrupted. Is there another way to implement that with lockless algorithms? Will that improve throughput, to support faster WAL writes and more concurrent threads? What do you sacrifice to implement lockless algorithms?

• Data lineage. This is metadata, to keep records of the origin of every piece of data stored in the database. Who created the data, when was it created, is it sensitive data like PII, who is authorized to view it? What about when data from different sources are combined? Many companies would be interested in a system to help track this, because often they have to create bespoke metadata in parallel to the RDBMS. If you could create features built into the RDBMS to track data lineage reliably and efficiently, that would be an achievement.

• Strong authentication. You didn't mention that your RDBMS is client/server. That is, can accept client connections over a network. If you add this, you should think about supporting TLS, and having a secure authentication system. Passwords are the traditional way, but that's old thinking. And at my last job, we required TLS certificates instead of passwords. Implementing these would teach you something new. Security is a deceptively challenging skill. How do you know your system is secure?

Thank you, that really helped. Even if my project is small, rebuilding these ideas from scratch has taught me a lot, and I should not ignore that.