Skip to content

Commit 38b2d76

Browse files
committed
feat: unify e2e tests, achieve full CLI/MCP feature parity
- Merge simulator tests into e2e/ directory with comprehensive coverage of all 116 tools against real Safari via iOS Simulator - Remove separate integration tests (proxy lifecycle now covered by e2e) - Add missing CLI commands: restart-iwdp, query-selector, get-outer-html, browser-extensions-enable, browser-extensions-disable - Split CLI into per-domain files for maintainability - Update mcp_test.go expectedTools to verify all 116 MCP tools - Simplify CI to two jobs: unit+smoke (ubuntu) and e2e (macos+simulator) - Bump version to 0.5.0
1 parent 11ae0b7 commit 38b2d76

26 files changed

Lines changed: 4951 additions & 1009 deletions

.claude-plugin/marketplace.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "iwdp-mcp",
3-
"version": "0.4.1",
3+
"version": "0.5.0",
44
"description": "iOS Safari debugging via ios-webkit-debug-proxy — MCP server with full WebKit Inspector Protocol support",
55
"owner": {
66
"name": "nnemirovsky"

.claude-plugin/plugin.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "iwdp-mcp",
3-
"version": "0.4.1",
3+
"version": "0.5.0",
44
"description": "iOS Safari debugging via ios-webkit-debug-proxy — MCP server with full WebKit Inspector Protocol support",
55
"mcpServers": {
66
"iwdp-mcp": {

.github/workflows/test.yml

Lines changed: 7 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -35,46 +35,6 @@ jobs:
3535
done
3636
3737
e2e:
38-
runs-on: ubuntu-latest
39-
steps:
40-
- uses: actions/checkout@v4
41-
42-
- uses: actions/setup-go@v5
43-
with:
44-
go-version-file: go.mod
45-
46-
- name: Run e2e tests
47-
run: go test ./e2e/ -v -count=1 -timeout=120s
48-
49-
integration:
50-
runs-on: macos-latest
51-
steps:
52-
- uses: actions/checkout@v4
53-
54-
- uses: actions/setup-go@v5
55-
with:
56-
go-version-file: go.mod
57-
58-
- name: Install ios-webkit-debug-proxy
59-
run: brew install ios-webkit-debug-proxy
60-
61-
- name: Run all tests with integration tag and coverage
62-
run: go test -tags=integration ./... -v -count=1 -timeout=120s -coverprofile=coverage-integration.out
63-
64-
- name: Integration coverage summary
65-
run: |
66-
total=$(go tool cover -func=coverage-integration.out | tail -1 | awk '{print $NF}')
67-
echo "## Integration Test Coverage (includes all tests)" >> "$GITHUB_STEP_SUMMARY"
68-
echo "" >> "$GITHUB_STEP_SUMMARY"
69-
echo "**Total: ${total}**" >> "$GITHUB_STEP_SUMMARY"
70-
echo "" >> "$GITHUB_STEP_SUMMARY"
71-
echo "| Package | Coverage |" >> "$GITHUB_STEP_SUMMARY"
72-
echo "|---------|----------|" >> "$GITHUB_STEP_SUMMARY"
73-
go tool cover -func=coverage-integration.out | grep -v "total:" | awk -F'\t+' '{print $1, $NF}' | sort -u -t' ' -k1,1 | while read -r pkg cov; do
74-
echo "| \`${pkg}\` | ${cov} |" >> "$GITHUB_STEP_SUMMARY"
75-
done
76-
77-
simulator:
7838
runs-on: macos-latest
7939
steps:
8040
- uses: actions/checkout@v4
@@ -110,21 +70,21 @@ jobs:
11070
eval "$(./scripts/sim-setup.sh)"
11171
echo "IWDP_SIM_WS_URL=$IWDP_SIM_WS_URL" >> "$GITHUB_ENV"
11272
113-
- name: Run simulator tests with coverage
114-
run: go test -tags=simulator ./... -v -count=1 -timeout=300s -coverprofile=coverage-simulator.out
73+
- name: Run e2e tests with coverage
74+
run: go test -tags=simulator ./e2e/ -v -count=1 -timeout=300s -coverprofile=coverage-e2e.out
11575

116-
- name: Simulator coverage summary
76+
- name: E2E coverage summary
11777
if: always()
11878
run: |
119-
if [ -f coverage-simulator.out ]; then
120-
total=$(go tool cover -func=coverage-simulator.out | tail -1 | awk '{print $NF}')
121-
echo "## Simulator Test Coverage" >> "$GITHUB_STEP_SUMMARY"
79+
if [ -f coverage-e2e.out ]; then
80+
total=$(go tool cover -func=coverage-e2e.out | tail -1 | awk '{print $NF}')
81+
echo "## E2E Test Coverage" >> "$GITHUB_STEP_SUMMARY"
12282
echo "" >> "$GITHUB_STEP_SUMMARY"
12383
echo "**Total: ${total}**" >> "$GITHUB_STEP_SUMMARY"
12484
echo "" >> "$GITHUB_STEP_SUMMARY"
12585
echo "| Package | Coverage |" >> "$GITHUB_STEP_SUMMARY"
12686
echo "|---------|----------|" >> "$GITHUB_STEP_SUMMARY"
127-
go tool cover -func=coverage-simulator.out | grep -v "total:" | awk -F'\t+' '{print $1, $NF}' | sort -u -t' ' -k1,1 | while read -r pkg cov; do
87+
go tool cover -func=coverage-e2e.out | grep -v "total:" | awk -F'\t+' '{print $1, $NF}' | sort -u -t' ' -k1,1 | while read -r pkg cov; do
12888
echo "| \`${pkg}\` | ${cov} |" >> "$GITHUB_STEP_SUMMARY"
12989
done
13090
fi

CLAUDE.md

Lines changed: 5 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,9 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
88
go build ./... # Compile (quick check)
99
make build # Compile to bin/iwdp-mcp + bin/iwdp-cli
1010
make install # Install both binaries via go install
11-
make test # Run all tests (go test ./... -v -count=1)
12-
make test-e2e # Run e2e tests (builds binaries, tests CLI + MCP server)
11+
make test # Run unit + smoke tests (go test ./... -v -count=1)
12+
make test-e2e # Run e2e tests (boots iOS Sim + iwdp, tests ALL tools)
1313
make test-coverage # Run tests with coverage → coverage.html
14-
make test-integration # Run integration tests (requires iwdp binary installed)
15-
make test-simulator # Run simulator tests (boots iOS Sim + iwdp, tests all tools)
1614
make sim-setup # Boot iOS Simulator + iwdp (prints IWDP_SIM_WS_URL)
1715
make sim-teardown # Shut down simulator + iwdp
1816
make fmt # gofumpt -w . (format all Go files)
@@ -89,15 +87,13 @@ Two binaries, one shared `internal/` package tree:
8987
- All WebKit protocol communication goes through `webkit.Client`
9088
- Tools return structured results; formatting is done by the caller (MCP or CLI)
9189
- Unit tests use mock WebSocket server from `internal/webkit/testutil/`
92-
- E2E tests in `e2e/` build actual binaries and test CLI help/error paths + MCP server JSON-RPC initialization
93-
- Integration tests (`-tags=integration`) use the real `ios_webkit_debug_proxy` binary
94-
- Simulator tests (`-tags=simulator`) boot iOS Simulator + iwdp and test all tools against real Safari
90+
- E2E tests in `e2e/` cover CLI binary smoke tests, MCP server JSON-RPC, and simulator-based tool tests (`-tags=simulator`) that test ALL tools against real Safari via iOS Simulator + iwdp
9591
- Error messages should be actionable (tell the user what to do)
9692
- Network/Console/Timeline use collector patterns: `Start()` registers event handlers, `Get*()` returns collected data
9793
- gorilla/websocket is not concurrent-write-safe — `Client.writeMu` mutex protects `conn.WriteMessage`
98-
- Simulator tests share a single WebSocket connection via `sync.Once` (`getSimClient`) — never create multiple connections to the same page
94+
- E2E simulator tests share a single WebSocket connection via `sync.Once` (`getSimClient` in `e2e/sim_helpers_test.go`) — never create multiple connections to the same page
9995
- Use `simOrigin()` helper to get the page's actual origin for storage tests — never hardcode origins
100-
- Use `t.Skipf` (not `t.Fatalf`) for features that may not be supported in all WebKit versions
96+
- Almost never use `t.Skipf`/`t.Skip`. Use `t.Fatalf`/`t.Fatal` instead. Only skip when a feature is specifically proven unsupported (e.g., `CSS.getAllStyleSheets` through iwdp Target routing). The env-var check in `getSimClient` is the only legitimate `t.Skip` in e2e tests.
10197

10298
## Git Conventions
10399

Makefile

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.PHONY: build test test-e2e test-integration test-simulator test-coverage \
1+
.PHONY: build test test-e2e test-coverage \
22
sim-setup sim-teardown lint fmt tidy install clean release release-snapshot
33

44
# Build
@@ -25,26 +25,20 @@ test-coverage:
2525
go test ./... -coverprofile=coverage.out
2626
go tool cover -html=coverage.out -o coverage.html
2727

28-
test-e2e:
29-
go test ./e2e/ -v -count=1 -timeout=120s
30-
31-
test-integration:
32-
go test -tags=integration ./... -v -count=1 -timeout=120s
33-
34-
# Simulator tests — boots iOS Simulator + iwdp, runs all tools against real Safari.
35-
# Usage: make test-simulator
36-
# or: make sim-setup && go test -tags=simulator ./... -v && make sim-teardown
28+
# E2E tests — boots iOS Simulator + iwdp, runs all tools against real Safari.
29+
# Usage: make test-e2e
30+
# or: make sim-setup && go test -tags=simulator ./e2e/ -v && make sim-teardown
3731
sim-setup:
3832
@eval "$$(./scripts/sim-setup.sh)" && echo "IWDP_SIM_WS_URL=$$IWDP_SIM_WS_URL"
3933

4034
sim-teardown:
4135
@./scripts/sim-setup.sh --teardown
4236

43-
test-simulator:
37+
test-e2e:
4438
@echo "==> Setting up iOS Simulator..."
4539
@eval "$$(./scripts/sim-setup.sh)" && \
46-
echo "==> Running simulator tests..." && \
47-
go test -tags=simulator ./... -v -count=1 -timeout=300s && \
40+
echo "==> Running e2e tests..." && \
41+
go test -tags=simulator ./e2e/ -v -count=1 -timeout=300s && \
4842
echo "==> Tearing down..." && \
4943
./scripts/sim-setup.sh --teardown
5044
@./scripts/sim-setup.sh --teardown 2>/dev/null || true

README.md

Lines changed: 8 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818

1919
<video src="https://github.com/user-attachments/assets/dcc0dc05-3a1f-4454-88a8-51ee28da9fd2" poster="https://github.com/user-attachments/assets/3901f7ff-6531-4866-b49e-cf5d17304327" autoplay loop muted playsinline></video>
2020

21-
100+ tools across all 27 WebKit Inspector Protocol domains — navigation, screenshots, DOM inspection, CSS, network interception, httpOnly cookies, JS debugging, heap snapshots, profiling, and more.
21+
100+ tools across all 27 WebKit Inspector Protocol domains — navigation, screenshots, DOM inspection, CSS, network interception, httpOnly cookies, JS debugging, heap snapshots, profiling, and more. Full feature parity between MCP server and CLI.
2222

2323
## Why?
2424

@@ -374,46 +374,32 @@ Animation, Canvas, LayerTree, Workers, Audit, Security (TLS certificates), and e
374374

375375
```bash
376376
make build # Build both binaries
377-
make test # Run all tests
377+
make test # Run unit + smoke tests
378+
make test-e2e # E2E tests (boots iOS Simulator, tests all tools)
378379
make test-coverage # Tests with coverage report
379380
make lint # golangci-lint
380381
make fmt # gofumpt formatting
381382
```
382383

383384
### Testing
384385

385-
Unit tests use a mock WebSocket server (`internal/webkit/testutil/`) that simulates the WebKit Inspector Protocol.
386+
Unit tests use a mock WebSocket server (`internal/webkit/testutil/`) that simulates the WebKit Inspector Protocol. E2E tests boot an iOS Simulator and run every tool against real Safari.
386387

387388
```bash
388-
# Unit tests (no device needed)
389+
# Unit + smoke tests (no device needed)
389390
make test
390391

391-
# E2E tests (builds binaries, tests CLI + MCP server JSON-RPC)
392+
# E2E tests — boots iOS Simulator + iwdp, tests ALL tools against real Safari
392393
make test-e2e
393394

394-
# Integration tests (requires iwdp binary installed)
395-
make test-integration
396-
397-
# Simulator tests — boots iOS Simulator + iwdp, tests ALL tools against real Safari
398-
make test-simulator
399-
```
400-
401-
#### iOS Simulator Tests
402-
403-
Simulator tests (`-tags=simulator`) boot an iOS Simulator, start `ios_webkit_debug_proxy` with the simulator's web inspector socket, and run every tool against a real Safari page. No physical device needed.
404-
405-
```bash
406-
# One-command: setup → test → teardown
407-
make test-simulator
408-
409395
# Or manually for debugging:
410396
make sim-setup # Prints IWDP_SIM_WS_URL
411397
export IWDP_SIM_WS_URL=ws://localhost:9222/devtools/page/1
412-
go test -tags=simulator ./... -v -run TestSim_Navigate
398+
go test -tags=simulator ./e2e/ -v -run TestSim_Navigate
413399
make sim-teardown
414400
```
415401

416-
> **Note:** Requires macOS with Xcode and `ios-webkit-debug-proxy` installed. GitHub Actions `macos-latest` runners have Xcode and iOS Simulator runtimes pre-installed.
402+
> **Note:** E2E tests require macOS with Xcode and `ios-webkit-debug-proxy` installed. GitHub Actions `macos-latest` runners have Xcode and iOS Simulator runtimes pre-installed.
417403
418404
### Project Structure
419405

0 commit comments

Comments
 (0)