Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

Hot takes on Devin, the AI software engineer | Tom Hipwell

Mar 18, 2024 - tomhipwell.co
The author discusses their impressions of Devin from Cognition, particularly admiring the user experience and likening it to a glimpse of a new era. They speculate that the performance improvement in the software engineering benchmarks could be due to a change in prompting technique, similar to the Self-Discover technique used by Google DeepMind. The author suggests that this style of prompting would be suitable for this type of reasoning task and wonders if there's a way to intercept the API calls to confirm this.

The author also considers whether the Cyborg AI style described by Ethan Mollick in a BCG paper would be a better fit for developer flow, suggesting it could be integrated into existing IDEs. They appreciate the pause and step buttons in the screenshots, comparing the interaction to turn-taking in a strategy game. They anticipate that if the performance uplift is indeed due to prompting, it could indicate the future of developer experience.

Key takeaways:

  • The author is impressed by the UX of Devin from Cognition and believes it could represent a new era.
  • There's speculation that the performance lift in the SWE benchmarks might be due to a change in prompting technique, similar to the Self-Discover technique by Google DeepMind.
  • The author suggests that the Cyborg AI style described by Ethan Mollick in a BCG paper might be a better fit for developer flow if integrated into existing IDEs.
  • If the performance uplift is indeed due to prompting, the author believes this could be a glimpse into the future of DX.
View Full Article

Comments (0)

Be the first to comment!