Skip to content

Commit 4df60d0

Browse files
Maffoochclaude
Maffooch
and
claude
committed
Add CLAUDE.md with module reorganization playbook
Documents a repeatable 10-phase process for reorganizing domain modules (finding, test, engagement, product, product_type) to match the dojo/url/ reference pattern. Includes service-layer extraction guidance to support the long-term goal of removing the classic UI and going fully API-based. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 8ead201 commit 4df60d0

1 file changed

Lines changed: 250 additions & 0 deletions

File tree

‎CLAUDE.md‎

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
# DefectDojo Development Guide
2+
3+
## Project Overview
4+
5+
DefectDojo is a Django application (`dojo` app) for vulnerability management. The codebase is undergoing a modular reorganization to move from monolithic files toward self-contained domain modules.
6+
7+
## Module Reorganization
8+
9+
### Reference Pattern: `dojo/url/`
10+
11+
All domain modules should match the structure of `dojo/url/`. This is the canonical example of a fully reorganized module.
12+
13+
```
14+
dojo/{module}/
15+
├── __init__.py # import dojo.{module}.admin # noqa: F401
16+
├── models.py # Domain models, constants, factory methods
17+
├── admin.py # @admin.register() for the module's models
18+
├── services.py # Business logic (no HTTP concerns)
19+
├── queries.py # Complex DB aggregations/annotations
20+
├── signals.py # Django signal handlers
21+
├── [manager.py] # Custom QuerySet/Manager if needed
22+
├── [validators.py] # Field-level validators if needed
23+
├── [helpers.py] # Async task wrappers, tag propagation, etc.
24+
├── ui/
25+
│ ├── __init__.py # Empty
26+
│ ├── forms.py # Django ModelForms
27+
│ ├── filters.py # UI-layer django-filter classes
28+
│ ├── views.py # Thin view functions — delegates to services.py
29+
│ └── urls.py # URL routing
30+
└── api/
31+
├── __init__.py # path = "{module}"
32+
├── serializer.py # DRF serializers
33+
├── views.py # API ViewSets — delegates to services.py
34+
├── filters.py # API-layer filters
35+
└── urls.py # add_{module}_urls(router) registration
36+
```
37+
38+
### Architecture Principles
39+
40+
**Long-term goal**: Remove the classic Django template UI and make the app fully API-based. The `ui/` subdirectory should be deletable with zero impact on `api/` or shared business logic.
41+
42+
**services.py is the critical layer**: Both `ui/views.py` and `api/views.py` call `services.py` for business logic. Services accept domain objects and primitives — never request/response objects, forms, or serializers.
43+
44+
**Backward-compatible re-exports**: When moving code out of monolithic files (`dojo/models.py`, `dojo/forms.py`, `dojo/filters.py`, `dojo/api_v2/serializers.py`, `dojo/api_v2/views.py`), always leave a re-export at the original location:
45+
```python
46+
from dojo.{module}.models import {Model} # noqa: F401 -- backward compat
47+
```
48+
Never remove re-exports until all consumers are updated in a dedicated cleanup pass.
49+
50+
### Current State
51+
52+
Modules in various stages of reorganization:
53+
54+
| Module | models.py | services.py | ui/ | api/ | Status |
55+
|--------|-----------|-------------|-----|------|--------|
56+
| **url** | In module | N/A | Done | Done | **Complete** |
57+
| **location** | In module | N/A | N/A | Done | **Complete** |
58+
| **product_type** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
59+
| **test** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
60+
| **engagement** | In dojo/models.py | Partial (32 lines) | Partial (views at root) | In dojo/api_v2/ | Needs work |
61+
| **product** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
62+
| **finding** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
63+
64+
### Monolithic Files Being Decomposed
65+
66+
These files still contain code for multiple modules. Extract code to the target module's subdirectory and leave a re-export stub.
67+
68+
- `dojo/models.py` (4,973 lines) — All model definitions
69+
- `dojo/forms.py` (4,127 lines) — All Django forms
70+
- `dojo/filters.py` (4,016 lines) — All UI and API filter classes
71+
- `dojo/api_v2/serializers.py` (3,387 lines) — All DRF serializers
72+
- `dojo/api_v2/views.py` (3,519 lines) — All API viewsets
73+
74+
---
75+
76+
## Reorganization Playbook
77+
78+
When asked to reorganize a module, follow these phases in order. Each phase should be independently verifiable.
79+
80+
### Phase 0: Pre-Flight (Read-Only)
81+
82+
Before any changes, identify all code to extract:
83+
84+
```bash
85+
# 1. Model classes and line ranges in dojo/models.py
86+
grep -n "class {Model}" dojo/models.py
87+
88+
# 2. Form classes in dojo/forms.py
89+
grep -n "class.*{Module}" dojo/forms.py
90+
grep -n "model = {Model}" dojo/forms.py
91+
92+
# 3. Filter classes in dojo/filters.py
93+
grep -n "class.*{Module}\|class.*{Model}" dojo/filters.py
94+
95+
# 4. Serializer classes
96+
grep -n "class.*{Model}" dojo/api_v2/serializers.py
97+
98+
# 5. ViewSet classes
99+
grep -n "class.*{Model}\|class.*{Module}" dojo/api_v2/views.py
100+
101+
# 6. Admin registrations
102+
grep -n "admin.site.register({Model}" dojo/models.py
103+
104+
# 7. All import sites (to verify backward compat)
105+
grep -rn "from dojo.models import.*{Model}" dojo/ unittests/
106+
107+
# 8. Business logic in current views
108+
# Scan dojo/{module}/views.py for: .save(), .delete(), create_notification(),
109+
# jira_helper.*, dojo_dispatch_task(), multi-model workflows
110+
```
111+
112+
### Phase 1: Extract Models
113+
114+
1. Create `dojo/{module}/models.py` with the model class(es) and associated constants
115+
2. Create `dojo/{module}/admin.py` with `admin.site.register()` calls (remove from `dojo/models.py`)
116+
3. Update `dojo/{module}/__init__.py` to `import dojo.{module}.admin # noqa: F401`
117+
4. Add re-exports in `dojo/models.py`
118+
5. Remove original model code (keep re-export line)
119+
120+
**Import rules for models.py:**
121+
- Upward FKs (e.g., Test -> Engagement): import from `dojo.models` if not yet extracted, or `dojo.{module}.models` if already extracted
122+
- Downward references (e.g., Product_Type querying Finding): use lazy imports inside method bodies
123+
- Shared utilities (`copy_model_util`, `_manage_inherited_tags`, `get_current_date`, etc.): import from `dojo.models`
124+
- Do NOT set `app_label` in Meta — all models inherit `dojo` app_label automatically
125+
126+
**Verify:**
127+
```bash
128+
python manage.py check
129+
python manage.py makemigrations --check
130+
python -c "from dojo.{module}.models import {Model}"
131+
python -c "from dojo.models import {Model}"
132+
```
133+
134+
### Phase 2: Extract Services
135+
136+
Create `dojo/{module}/services.py` with business logic extracted from UI views.
137+
138+
**What belongs in services.py:**
139+
- State transitions (close, reopen, status changes)
140+
- Multi-step creation/update workflows
141+
- External integration calls (JIRA, GitHub)
142+
- Notification dispatching
143+
- Copy/clone operations
144+
- Bulk operations
145+
- Merge operations
146+
147+
**What stays in views:**
148+
- HTTP request/response handling
149+
- Form instantiation and validation
150+
- Serialization/deserialization
151+
- Authorization checks (`@user_is_authorized`, `user_has_permission_or_403`)
152+
- Template rendering, redirects
153+
- Pagination, breadcrumbs
154+
155+
**Service function pattern:**
156+
```python
157+
def close_engagement(engagement: Engagement, user: User) -> Engagement:
158+
engagement.active = False
159+
engagement.status = "Completed"
160+
engagement.save()
161+
if jira_helper.get_jira_project(engagement):
162+
dojo_dispatch_task(jira_helper.close_epic, engagement.id, push_to_jira=True)
163+
return engagement
164+
```
165+
166+
Update UI views and API viewsets to call the service instead of containing logic inline.
167+
168+
### Phase 3: Extract Forms to `ui/forms.py`
169+
170+
1. Create `dojo/{module}/ui/__init__.py` (empty)
171+
2. Create `dojo/{module}/ui/forms.py` — move form classes from `dojo/forms.py`
172+
3. Add re-exports in `dojo/forms.py`
173+
174+
### Phase 4: Extract UI Filters to `ui/filters.py`
175+
176+
1. Create `dojo/{module}/ui/filters.py` — move module-specific filters from `dojo/filters.py`
177+
2. Shared base classes (`DojoFilter`, `DateRangeFilter`, `ReportBooleanFilter`) stay in `dojo/filters.py`
178+
3. Add re-exports in `dojo/filters.py`
179+
180+
### Phase 5: Move UI Views/URLs into `ui/`
181+
182+
1. Move `dojo/{module}/views.py` -> `dojo/{module}/ui/views.py`
183+
2. Move `dojo/{module}/urls.py` -> `dojo/{module}/ui/urls.py`
184+
3. Update URL imports:
185+
- product: update `dojo/asset/urls.py`
186+
- product_type: update `dojo/organization/urls.py`
187+
- others: update the include in `dojo/urls.py`
188+
189+
### Phase 6: Extract API Serializers to `api/serializer.py`
190+
191+
1. Create `dojo/{module}/api/__init__.py` with `path = "{module}"`
192+
2. Create `dojo/{module}/api/serializer.py` — move from `dojo/api_v2/serializers.py`
193+
3. Add re-exports in `dojo/api_v2/serializers.py`
194+
195+
### Phase 7: Extract API Filters to `api/filters.py`
196+
197+
1. Create `dojo/{module}/api/filters.py` — move `Api{Model}Filter` from `dojo/filters.py`
198+
2. Add re-exports
199+
200+
### Phase 8: Extract API ViewSets to `api/views.py`
201+
202+
1. Create `dojo/{module}/api/views.py` — move from `dojo/api_v2/views.py`
203+
2. Add re-exports in `dojo/api_v2/views.py`
204+
205+
### Phase 9: Extract API URL Registration
206+
207+
1. Create `dojo/{module}/api/urls.py`:
208+
```python
209+
from dojo.{module}.api import path
210+
from dojo.{module}.api.views import {ViewSet}
211+
212+
def add_{module}_urls(router):
213+
router.register(path, {ViewSet}, path)
214+
return router
215+
```
216+
2. Update `dojo/urls.py` — replace `v2_api.register(...)` with `add_{module}_urls(v2_api)`
217+
218+
### After Each Phase: Verify
219+
220+
```bash
221+
python manage.py check
222+
python manage.py makemigrations --check
223+
python -m pytest unittests/ -x --timeout=120
224+
```
225+
226+
---
227+
228+
## Cross-Module Dependencies
229+
230+
The model hierarchy is: Product_Type -> Product -> Engagement -> Test -> Finding
231+
232+
Extract in this order (top to bottom) so that upward FKs can import from already-extracted modules. The recommended order is: product_type, test, engagement, product, finding.
233+
234+
For downward references (e.g., Product_Type's cached properties querying Finding), always use lazy imports:
235+
```python
236+
@cached_property
237+
def critical_present(self):
238+
from dojo.models import Finding # lazy import
239+
return Finding.objects.filter(test__engagement__product__prod_type=self, severity="Critical").exists()
240+
```
241+
242+
---
243+
244+
## Key Technical Details
245+
246+
- **Single Django app**: Everything is under `app_label = "dojo"`. Moving models to subdirectories does NOT require migration changes.
247+
- **Model discovery**: Triggered by `__init__.py` importing `admin.py`, which imports `models.py`. This is the same chain `dojo/url/` uses.
248+
- **Signal registration**: Handled in `dojo/apps.py` via `import dojo.{module}.signals`. Already set up for test, engagement, product, product_type.
249+
- **Watson search**: Uses `self.get_model("Product")` in `apps.py` — works via Django's model registry regardless of file location.
250+
- **Admin registration**: Currently at the bottom of `dojo/models.py` (lines 4888-4973). Must be moved to `{module}/admin.py` and removed from `dojo/models.py` to avoid `AlreadyRegistered` errors.

0 commit comments

Comments
 (0)