Comment on: improvements to novelty
Repo: karpathy/autoresearch by mkemka
Currently I can only talk to the experiments I made in the fork (https://github.com/mkemka/autoresearch/blob/master/spiritualguidance.md). There are two competing agents that argue and generate a combined directive that is used to alter the program.md for the next run. The history is stored in the spiritual guidance.md and used as a working memory. So to actually measure the utility I would need to see if there is actually novelty or variance of ideas from this approach and if in the long term the loss is lower compared to a single agent.
GitHub Issue
SaaS Metrics