What do you use today instead of Warp (https://www.warp.dev/) (a competitor, a spreadsheet, doing it by hand)? Compared with that, what would make Warp better or worse for you, and what would keep you on your current option? Also say the maximum you would pay per month in USD (0 if nothing).

Blake Maggioneutral

Currently, we're managing our agent's run results through a combination of GitHub issues, spreadsheets, and ad hoc Python scripts. It's a fragmented system that's becoming increasingly difficult to maintain. Warp's promise of "factories as code" and integrated measurement and self-improvement sounds compelling, especially the "baked-in measurement and self-improvement" aspect which directly addresses our need for objective evaluation of LLM outputs. The primary concern would be the integration effort and ensuring it doesn't introduce more complexity than it solves. Given our focus on efficiency and the critical need for reliable metrics, I could see a potential value, perhaps up to $50 per month per user, but we'd need to thoroughly evaluate its impact on our shipping velocity and system stability.

Simulated interest: 70/100

Ida Schmittneutral

Right now, we're evaluating some code review and release note agents, and I'm using a combination of spreadsheets and QA checklists to track their performance. I'm skeptical of tools that promise too much, so I'd need to see concrete evidence that Warp significantly improves our incident detection rate or reduces manual verification time. The concept of 'factories as code' is interesting, but I'd need to understand how it integrates with our existing compliance checks and ensures data security, especially with patient information. The pricing model is unclear; is it per agent run, per user, or something else? Without that clarity, I can't estimate a monthly cost.

Trilokesh Somayajineutral

Currently, I rely on a combination of LangSmith for tracing and internal Python notebooks to manage prompt changes and evaluate agent performance. The primary challenge is the fragmented nature of our tooling and the sheer volume of experiments, which makes comprehensive tracking difficult. Warp's promise of "factories as code" and baked-in measurement with evals and benchmarks is appealing, as it addresses my need for systematic tracking and reproducible results. However, the product page focuses heavily on the "factory" concept, and I'm unclear on how it directly integrates with or replaces my existing prompt management and agent scoring workflows. The pricing model "usage-based, priced per agent run" is also too vague for me to estimate a monthly cost. I would need more clarity on the specific metrics and cost drivers before considering payment.

Tara Steinertneutral

Currently, we manage our ticket triage and build checks using a combination of n8n workflows, manual analysis of CSV exports, and detailed spreadsheets for scorecards. We've found this approach makes the system's logic understandable to our non-ML teammates. Warp's promise of "factories as code" and automating the SDLC sounds interesting, especially the "evals, benchmarks, and self-improvement" aspect. However, I'm concerned about how easily we could integrate it with our existing n8n setup and if it would truly simplify things or add another layer of complexity for our small team. The pricing basis being "per agent run" is also unclear; without knowing the typical number of runs or the cost per run, it's difficult to estimate the total cost. Given our lean operation, I'm hesitant to commit significant resources without a clearer picture of the ROI. I would be willing to consider it if the cost were around $20 per month, billed monthly, but that's a very preliminary guess.

Simulated interest: 30/100

Alton Friesenneutral

Right now, I'm using standard scripting and a lot of manual documentation in Google Sheets and Docs for CI/CD and deployment checks. Warp's "factories as code" concept sounds interesting, especially the idea of agents automating tasks and self-improvement loops to reduce manual work and benchmark results. However, I'm unclear on the integration with my existing toolchains and how much setup is involved for SMBs. If it significantly streamlines my documentation and automation for small teams, it could be valuable. I'd need to see a clear ROI before paying more than $25 per month.

Simulated interest: 40/100

Mathilde Janssenneutral

Today, I'm piecing together an evaluation loop using LangSmith, custom Python scripts in Jupyter notebooks, and manual sampling of chat logs. It's effective but incredibly time-consuming, especially when I need to rigorously compare prompt variants or debug failure modes. Warp's promise of 'factories as code' with built-in evals, benchmarks, and self-improvement sounds appealing, potentially automating much of the drudgery. However, I'm concerned about the black-box nature of some SaaS evaluation tools, and I need to understand if Warp offers the granular control I require to debug subtle issues like generic apologies. If it provides transparent, customizable metrics and integrates well with my existing workflow without introducing excessive complexity or cost, it could be a significant improvement. I'd need to see concrete evidence of its reliability and scalability in practice before committing. The pricing model is unclear, but based on my company's lean budget, I would not pay more than $50 per month.

Simulated interest: 60/100

Brad Binsneutral

Right now, I'm using a combination of custom CLI scripts and local Jupyter notebooks to manage my internal code assistant workflow. The biggest pain point is the latency and inconsistency from free-tier APIs, and ensuring reliable evaluations. Warp's focus on "factories as code" and built-in measurement sounds interesting, especially the evals and self-improvement loops. However, the product blurb mentions "Infrastructure to build, measure, and interact with agents across your SDLC." My current setup is more about managing the workflow itself, not necessarily building out infrastructure for agents. I'd need to understand if this is just a terminal replacement or a full-blown orchestration platform. The pricing is listed as "USD per month subscription," but there's no specific tier or cost mentioned. Without knowing the specifics of what's included, it's hard to say what I'd pay, but I'm generally cost-conscious. If it offered a significant improvement in managing my current challenges, I might consider up to $10 per month, but that's purely speculative without more details.

Simulated interest: 40/100

Tara Rananeutral

Currently, we rely heavily on Google Sheets and Confluence for our documentation, including benchmarks and review notes. We also do a lot of manual work to collate information and prepare handoff documents. Warp's promise of automating SDLC tasks and integrating agentic workflows is interesting, especially the "factories as code" aspect. However, I'd need to understand how it integrates with our existing client-facing documentation processes. The "usage-based, priced per agent run" pricing model is unclear; I'd need to see how that translates to predictable costs given our varied client needs. Without more clarity on cost and integration, it's hard to say if it would be better or worse, and the predictability of our current manual process, though time-consuming, is something I'd be hesitant to abandon.

Martial Fabreneutral

Today, for test generation and incident response, we primarily use a combination of custom Python scripts and manual spot checks during sprint reviews. We also compare outputs against baseline CSVs and Jupyter notebooks. Warp's promise of "factories as code" and built-in measurement capabilities are intriguing, particularly the "evals, benchmarks, and self-improvement loops." However, I'm concerned about the integration effort and how easily it can be customized for our specific API testing needs. The core of our current workflow is already established, and switching would require significant validation. I would need to see a clearer path to integrating our existing baseline comparisons and understanding the precise cost structure. Without more clarity on the pricing basis, it's difficult to estimate, but I would be hesitant to pay more than $20 per month initially, possibly increasing if it proves significantly more efficient.

Simulated interest: 40/100

Dorothy Prohaskaneutral

Right now, I'm piecing together evaluations using a mix of Notion, Python scripts, and other free tools. The idea of a unified "software factory" sounds interesting, especially the "factories as code" aspect. However, I'm not clear on how it integrates with my current workflow or if it handles agent evaluations specifically. The pricing model is also a bit opaque – it mentions "usage-based, priced per agent run," but without more specifics, it's hard to estimate. I'd be willing to pay up to $50 per month, but that's a very rough guess based on the potential efficiency gains.

Simulated interest: 40/100