A week ago, we upgraded Delphi to GPT-4, OpenAI's newest and most powerful large language model. For us, this update is all about making company data accessible to everyone, not just analysts and developers.
You might have heard the buzz and be wondering how much better GPT-4 really is. Now that we’ve had a chance to play with it, I wanted to share my initial thoughts.
Where GPT-4 shines
1. Impressive query understanding: with GPT-4, queries are more accurate and efficient. GPT-4 is better at understanding what you're asking, so you don't need to be super precise with your questions. This means less back-and-forth and a smoother experience.
2. Smart data interpretation: GPT-4 is much better at understanding the data we return, which means you'll get more accurate and relevant insights for your decision-making.
3. It can finally do math! : previous versions like ChatGPT-3.5 weren’t very good at math. You might have seen stuff like this:
But GPT-4 can handle numbers like a champ (or at least a reasonably smart intern), making it easy to answer questions that require calculations. Think period-over-period analyses, percent changes, and table calculations.
Here’s an example:
The Challenges
While GPT-4 is pretty amazing, there are some downsides:
Speed: GPT-4 might be a tad slower than ChatGPT when processing queries. Overall, we’ve found this isn’t a huge deal (it’s still faster than waiting on a human!) and we're working on optimizing our platform to keep things running as smoothly as possible.
Codex deprecation: OpenAI turned off code-davinci-002 (Codex), their code-specific model that we used for the query generation step, around the same time we got GPT-4 access. This isn’t a downside of GPT-4 per se, but it meant we needed to re-work some of our prompts and post-processing (and quickly—they only gave 3 days’ notice!). Overall, GPT-4 seems about as good at code generation as Codex was, but much better than ChatGPT.
Conclusion
GPT-4 is a leap forward, and by upgrading to it, Delphi is taking a big step towards making data more user-friendly for everyone. And when you connect Delphi to dbt, Metabase or Lightdash, data teams can help business users run queries with ease.
Yes, there are some challenges, but we're committed to making sure you get the best experience possible. Stay tuned for more updates, as we continue to enhance the world of natural language data querying. Thanks for reading!
If you use the dbt semantic layer, Lightdash or Metabase and are interested in Delphi, reach out to me or sign up at delphihq.com