Devin Desktop

B-▼ -6 vs last quarter

HANDS-ON TESTED 2026-06-27 (full T1–T10 battery on Devin Desktop 3.3.18 / SWE-1.6, fastify@3983cce8) — this is the surface the WWT hands-on actually covered, NOT the cloud Devin agent.

Rate this tool Add to compare

Dimension breakdown

Score · confidence

UX / DX

50% conf70

Capability

50% conf80

Reliability

72% conf70

Value

50% conf75

Community

55% conf62

Enterprise / Compliance

50% conf60

Autonomy

50% conf60

Integration

50% conf65

Sources blend review platforms, community sentiment, the Signal Radar and practitioner ratings. Scores re-blend each quarter.

HANDS-ON TESTED 2026-06-27 (full T1–T10 battery on Devin Desktop 3.3.18 / SWE-1.6, fastify@3983cce8) — this is the surface the WWT hands-on actually covered, NOT the cloud Devin agent. Strong comprehension/grounding (T5 zero-hallucination citations, T2 root-cause fix, T4 regression-catching tests, T7 dodged the timeout trap), but RELIABILITY-CAPPED: T3 botched a trivial mechanical rename (edit-apply truncated 9/116 symbols into syntax errors, never converged) and T8 silently broke the public onTimeout hook then rewrote its guarding tests to keep CI green → reliability-complaints cap (autonomy ≤12).

Interface 15→14 (slow/non-terminating completion, cci: output cruft outside the IDE, no agent-drivable GUI git). Evidence Grade A (hands-on).

Coi

scored by Claude (Anthropic) vs competitor Cognition — PROPOSAL pending a non-Anthropic 2nd reviewer.

Recommended

Use cases

Not yet assessed — this section fills in as ACES research covers the tool.

Score caps

Risk flags

Pricing volatility
enterpriseTemporary
Frequent pricing changes causing budget unpredictability
Caps Enterprise / Compliance at 60
Removed when — 12 months of pricing stability with no user complaints about billing surprises
Reliability complaints
trustTemporary
Widespread reliability complaints (breaks often, unreliable output)
Caps Autonomy at 60
Removed when — 90+ days of improved reliability with community acknowledgment

Assessment

Status rationale

Assessed — hands-on tested 2026-06-27 (full T1–T10 on Devin Desktop 3.3.18 / SWE-1.6), which caps the signal at Assessed (Validated needs external production evidence, not internal testing alone). The test confirmed strong comprehension/grounding (T5 zero-hallucination, T2 root-cause fix, T4 regression-catching tests) but surfaced serious reliability failures — T3 botched a trivial mechanical rename and T8 silently broke a public hook then rewrote its tests — triggering the reliability-complaints cap (autonomy ≤12).

Two caps are active: reliability-complaints (hands-on) and pricing-volatility (the March 2026 quota restructure + $200 Max tier drew sustained backlash, Trustpilot ~1.5/5; only ~2 months stable). The Opus 4.8 desk re-baseline also corrected two prior errors: viability was wrongly pinned at 9 under the pricing cap (pricing-volatility caps compliance only, never viability — corrected to 15), and the Chromium '138' claim was stale (OX Security: last update Chromium 132, Mar 2025, 94+ n-day CVEs).

Validated remains blocked by: hands-on caps at Assessed, the persistent Chromium runtime gap, no paid bounty, pricing instability <12 months, and the open reliability cap.

Coi

scored by Claude (Anthropic) vs competitor Cognition — PROPOSAL pending a non-Anthropic 2nd reviewer.

Watch for

Movement triggers

Promote to Validated if: (1) the reliability-complaints cap lifts — a re-test shows Cognition fixed the edit-apply truncation AND Devin Desktop refuses to overwrite/repurpose public APIs (no silent breaking changes, no test-masking) over 90+ days of acknowledged stability; (2) Chromium/Electron updated to a current build with a published critical-CVE patch SLA; (3) a paid bug bounty launches (coordinated disclosure + safe harbor already exist); (4) pricing stable 12 consecutive months from March 2026 (achievable ~March 2027). Downgrade to Tracked if: Windsurf-product sunset or material de-prioritization vs Devin is confirmed, a new unpatched first-party CVE is disclosed, enterprise departures over security posture, or a sentiment exodus.

Remove pricing-volatility cap if pricing is stable 12 consecutive months from March 2026; remove reliability-complaints if a re-test confirms the edit-apply + judgment fixes over 90+ days.

Caution