Feedback from evaluating spec-kit in our pre-alpha IDP project

[ourchitectureio]

Hi spec-kit team,

First, thank you for the work you have put into spec-kit.

With the guidance and help from @ericis (https://www.linkedin.com/in/iseric), we performed an evaluation of this project for possible use in the IDP project. We also documented that evaluation publicly here:

IDP ADR: spec-kit evaluation

We wanted to share feedback that is meant to be positive, constructive, and useful. Overall, we came away with respect for the vision behind spec-kit and appreciation for the practical direction it is taking.

What stood out positively

1. Specification is treated as a real delivery artifact

One of the strongest aspects of spec-kit is that it appears to treat specification as something closer to a working asset, not just documentation written after decisions have already been made.

That matters. Many teams say they are spec-driven, but the specification ends up being static prose. spec-kit feels closer to helping teams create artifacts that can guide implementation more directly.

2. It encourages shared understanding earlier

That is valuable for reducing ambiguity before code grows, especially when multiple contributors or AI agents are involved.

3. It improves handoff quality

There is real value in having something stronger in the middle between idea and implementation. This can reduce interpretation drift across product, engineering, and AI-assisted workflows.

4. It is pointed at an important gap

There is a real industry need for better structure between intent, planning, validation, and delivery. spec-kit is clearly aiming at a meaningful problem.

Where this aligned closely with the IDP project

There was meaningful overlap between spec-kit and our own internal direction.

In the IDP project, we are intentionally pushing toward:

• clearer intent before implementation
• contract-oriented thinking
• stronger separation between specification, contract, and implementation
• artifacts that can guide both humans and AI agents
• better traceability from idea to validation

Because of that, spec-kit felt relevant very quickly.

At a philosophy level, we saw strong alignment in areas such as:

• reducing ambiguity early
• making work easier to review before implementation expands
• improving the path from requirement to implementation
• providing better structure for AI-assisted work than loose prompts alone

Where alignment became weaker

Our project also has goals that extend beyond specification as a planning artifact.

The IDP is trying to support a broader delivery system where specifications need to connect more directly to:

• contract tests
• multi-stack implementations
• observable runtime behavior
• security and privacy constraints
• authorization boundaries
• local development ergonomics
• agent-facing and human-facing execution surfaces
• progressive adoption across different organizational maturity levels

That is where we started to see some possible gaps, or at least areas where spec-kit may not be aiming to go as far as we need.

Potential opportunities to consider

Stronger pathways from specification to executable contract

For us, the biggest leverage comes when a spec does not just help people understand the work, but also anchors validation across implementations.

A clearer path from spec artifact to contract tests, schemas, fixtures, or conformance checks would make this even more powerful.

Better support for sequencing and dependency modeling

In platform work, some capabilities must exist before others can be safely introduced. Dependency-aware delivery matters.

Richer ways to model prerequisites, trust boundaries, and rollout order could increase usefulness for architecture and platform teams.

Governance and operational constraints as first-class concerns

Many spec systems are strongest at feature behavior and weaker at cross-cutting constraints.

In our case, things like auth, privacy, telemetry, auditability, and policy boundaries are not side concerns. They are structural requirements.

Multi-surface delivery support

We increasingly work across web UI, APIs, background automation, and AI-agent tool surfaces.

A specification approach that can unify intent across multiple surfaces becomes much more strategic.

Local development and contributor operability

Adoption gets much easier when teams can move from concept to realistic local execution without heavy setup cost.

Good ideas often lose momentum when local dev becomes too expensive. Developer experience is part of whether a specification approach survives contact with reality.

AI-agent reliability and constraint handling

This may be one of the most promising areas.

In our experience, AI agents perform better when sequencing is explicit, constraints are clear, non-goals are visible, and success states are observable. spec-kit seems well-positioned to help here, and more structure in this area could be especially compelling.

Progressive adoption patterns

Not every team can adopt a full specification discipline at once.

It helps when a project supports crawl, walk, run adoption. Lightweight planning first, then stronger contracts, then richer governance and conformance can be a much more realistic path for many organizations.

A framing that may help

One way we would describe this is:

spec-kit already appears helpful for defining what should be built.

The next level of leverage may be helping teams prove:

• what must be true
• how it will be validated
• what constraints must hold across implementations
• what operational realities must not be ignored

That distinction matters a great deal in platform engineering, enterprise delivery, and AI-assisted workflows.

Where this may simply reflect project-goal differences

Some of the above may not be gaps at all. They may simply reflect different goals.

Our IDP project is trying to connect planning, contracts, implementation, runtime signals, and AI interaction surfaces as one broader system.

spec-kit may intentionally be focused on an earlier and narrower slice of that lifecycle. If so, that is completely reasonable. Focus is often a strength.

So this feedback is not meant as "spec-kit should become our framework."

It is more:

spec-kit is pointed in a direction we respect, and there are adjacent areas that would become especially compelling if they fit your roadmap.

What we would be excited to see

The kinds of evolution that would make spec-kit even more compelling for projects like ours include:

• stronger spec-to-contract pathways
• better modeling of dependencies and rollout order
• richer support for auth, privacy, telemetry, and policy constraints
• clearer affordances for AI-agent execution and validation
• progressive adoption patterns for real teams
• better support for systems that span multiple delivery surfaces

Closing thought

We appreciated the chance to evaluate spec-kit because it is clearly trying to improve an important part of software delivery that many teams still handle poorly.

There is real signal in the vision.

Our takeaway was not that spec-kit missed the mark. It was that it already points toward something useful, and with the right scope choices it could become even more valuable for teams working across specifications, contracts, runtime systems, and AI-assisted delivery.

Thanks again for building and sharing it.

[ericis]

Well said! The spec-kit project has real opportunities ahead.

[mnriem]

Thank you!