Links Justin Sends 5/2

This past week I was fortunate enough to share some of our exciting AI research at BetterFutureLabs with the AI research community. Let’s unpack some of the insights and lessons shared.

🛠️ Principles for Building Model-Agnostic AI Systems — I shared some of the best practices we’ve learned building model-agnostic AI systems in Towards AI, a leading AI publication.

So, what are model-agnostic AI systems? They’re systems designed to be compatible with future models as they continue to evolve and add new levels of competence and capability — without needing to rebuild your infrastructure. Instead of locking into one provider or model architecture, you create a flexible stack that can continuously integrate the best tools available.

The blueprint for adaptable AI system architectures focuses on decoupling task logic from models, treating models as specialists, building modular systems where components can be swapped or upgraded independently, and embedding observability so you can track system performance over time.

Evaluating and testing models is where the rubber meets the road. It’s not enough to look at benchmarks — you need to assess real-world behavior in your systems. That means running A/B tests, using tools like PromptFoo to evaluate prompt and agent workflows, measuring latency and cost, and understanding whether a model’s performance holds up under real world conditions.

🤖 Building Multi-Agent Systems for Investment Analysis (AG2 Talk) — I also had the opportunity to give a talk for AG2, the leading open-source multi-agent system framework we have been actively using and contributing to since 2023.

I shared how we are using coordinated AI agents to dramatically accelerate our investment research and validation process and some best practices we’ve learn along our path of building it.

Each agent needs to have a clear, distinct responsibility — whether it’s planning, scoring, data gathering, or validation — to avoid overlap and keep the system operating smoothly. Without defined roles, agents can easily duplicate efforts or work against each other, which undercuts the system’s effectiveness.

It’s critical to design the system so agents can seamlessly share information, align on responsibilities, and understand how decisions get made. When agents know not only their own role but how their work fits into the larger process, you create a coordinated, reliable team instead of a loose collection of bots.

Consensus and decision-making are at the heart of a successful multi-agent system. Defining how agents converge on recommendations, how disagreements are resolved, and when the system is ready to finalize an output has been essential to delivering analyst-level quality.

Observability is what keeps the system improving. By tracking how agents interact, where breakdowns happen, and how decisions are reached, we’ve been able to fine-tune coordination and build confidence in the system’s outputs as it scales.

Hopefully you found these tips helpful! Feel free to forward this to your AI research and engineering teams — and as always, if you have any questions or want to discuss these topics, I’m more than happy to connect.

Have a great weekend!

-Justin

aka the guy with great AI links

Co-founder & Head of Technology @ BetterFutureLabs

[email protected] | LinkedIn