Harden data-update-partners workflow (#270)

* Harden data-update-partners workflow and add Last_Name fallback

Add error handling, input validation, and safety checks to prevent
empty data from overwriting good data. Add Last_Name field support
for maintainers, falling back to Last_Name when First_Name is empty.

* Fix curl error handling to work with set -euo pipefail

Use conditional capture (if ! response=$(curl ...)) instead of
$? check, which was unreachable under set -e.

* Add days since last commit to maintainers JSON

Call days_since_last_commit.py per maintainer to enrich the JSON
with a Days_since_last_commit field. Add --days-only flag to the
script for machine-readable output.

* Limit days-since-last-commit to pinned repos only

Use GitHub GraphQL API to fetch only pinned repositories instead
of all org repos, reducing API calls and avoiding errors from
large or problematic repositories.

* Retry on server errors instead of warning

Add exponential backoff retry (up to 3 attempts) for 500+ errors
when fetching commit dates, silently returning None if all retries
are exhausted.

* Use Search Commits API for days since last commit

Replace per-repo iteration with a single search API call per
maintainer (author:{user} org:{org}), drastically reducing API
usage and eliminating server error warnings.

* Handle 403 rate limits with retry and backoff

Fix Accept header, add retry with Retry-After for 403 responses,
handle 422 validation errors, and return None instead of crashing
when all retries are exhausted.

* Add verbose logging for maintainer processing

* Skip maintainers without GitHub profile, show name in logs

Display First_Name and Last_Name alongside username in processing
logs. Skip entries with null/empty GitHub field with a warning
showing the maintainer's name.

* Improve days-since-last-commit log output for no-commit users

* Fix rate limiting: increase retries and add delay between calls

Increase retry attempts to 5 with longer backoff (10s, 20s, 40s...),
and add 2s delay between maintainer lookups to avoid triggering
GitHub search API secondary rate limits.

* Track last activity (commits, issues, comments) not just commits

Search across commits, opened issues/PRs, and comments to find the
most recent activity date per maintainer. Rename JSON field to
Days_since_last_activity. Add 1s delay between search API calls.

* Remove comment search from activity check

The commenter search used issue updated_at which reflects when
anyone last touched the issue, not when the user commented,
giving false recent activity. Only track commits and opened
issues/PRs which have reliable dates.

* Fail job on partner fetch error instead of continuing with partial data

* Fail fast on maintainers fetch error with clear error message

* Make avatar lookup best-effort to avoid failing under pipefail

* Distinguish lookup failures from no-activity in days_since script

Return ERROR sentinel and exit 1 on API failures (rate limit
exhaustion, HTTP errors, network errors). Reserve -1 for genuine
no-activity. Workflow catches ERROR and falls back to -1 with a
warning instead of crashing.

Author: igorpecovnik

.github/workflows/data-update-partners-data.yml

@@ -22,7 +22,7 @@ jobs:
         uses: actions/checkout@v6
         with:
           repository: armbian/armbian.github.io
-          fetch-depth: 0
+          fetch-depth: 1
           clean: false
           path: armbian.github.io
 
@@ -33,26 +33,38 @@ jobs:
           REFRESH_TOKEN: ${{ secrets.ZOHO_REFRESH_TOKEN }}
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
+          set -euo pipefail
+
           ACCESS_TOKEN=$(curl -sH "Content-type: multipart/form-data" \
-            -F refresh_token=$REFRESH_TOKEN \
-            -F client_id=$CLIENT_ID \
-            -F client_secret=$CLIENT_SECRET \
+            -F "refresh_token=$REFRESH_TOKEN" \
+            -F "client_id=$CLIENT_ID" \
+            -F "client_secret=$CLIENT_SECRET" \
             -F grant_type=refresh_token \
             -X POST https://accounts.zoho.eu/oauth/v2/token \
             | jq -r '.access_token')
 
+          if [ -z "$ACCESS_TOKEN" ] || [ "$ACCESS_TOKEN" = "null" ]; then
+            echo "::error::Failed to obtain Zoho access token"
+            exit 1
+          fi
+
           echo "Access token obtained."
 
           # Fetch all partners and combine into single JSON file
           > partners.json
 
           for partner_type in Platinum Gold Silver; do
-            curl --silent --request GET \
+            if ! response=$(curl --silent --fail --request GET \
               --url "https://www.zohoapis.eu/bigin/v2/Accounts/search?fields=Website,Company_slug,Description,Account_Name,Email,Partnership_Status,Promoted&criteria=((Partnership_Status:equals:${partner_type}%20Partner)and(Promoted:equals:true))" \
-              --header "Authorization: Zoho-oauthtoken $ACCESS_TOKEN" \
+              --header "Authorization: Zoho-oauthtoken $ACCESS_TOKEN"); then
+              echo "::error::Failed to fetch ${partner_type} partners"
+              exit 1
+            fi
+
+            echo "$response" \
               | jq -c ".data[] | select(.Company_slug != null and .Company_slug != \"\") | {
                   Account_Name,
-                  logo_url: (\"https://cache.armbian.com/images/vendors/150/\(.Company_slug)-border.png\"),
+                  logo_url: (\"https://cache.armbian.com/images/vendors/150/\(.Company_slug).png\"),
                   Website,
                   Description,
                   Partnership_Status: .Partnership_Status
@@ -63,37 +75,37 @@ jobs:
           jq -s . < partners.json > partners.json.tmp && mv partners.json.tmp partners.json
           jq . -S < partners.json > partners.json.tmp && mv partners.json.tmp partners.json
 
+          # Validate partners data
+          partner_count=$(jq 'length' partners.json)
+          if [ "$partner_count" -lt 1 ]; then
+            echo "::error::Partners JSON is empty, refusing to commit"
+            exit 1
+          fi
+          echo "Fetched $partner_count partners."
+
           # Display content in step summary as markdown sections
           echo "# Partners" >> $GITHUB_STEP_SUMMARY
           echo "" >> $GITHUB_STEP_SUMMARY
 
-          # Platinum Partners
-          echo "## Platinum" >> $GITHUB_STEP_SUMMARY
-          echo '<p align=left>' >> $GITHUB_STEP_SUMMARY
           TIMESTAMP=$(date +%s)
-          jq -r --arg ts "$TIMESTAMP" '.[] | select(.Partnership_Status == "Platinum Partner") |
-            "<a href='\''\(.Website)'\''><img src='\''\(.logo_url)?v=\($ts)'\'' width='\''150'\'' alt='\''\(.Account_Name)'\'' style=\"display:block; border:0; height:auto;\"></a>"' \
-            partners.json | tr '\n' ' ' >> $GITHUB_STEP_SUMMARY
-          echo '</p>' >> $GITHUB_STEP_SUMMARY
-          echo "" >> $GITHUB_STEP_SUMMARY
-
-          # Gold Partners
-          echo "## Gold" >> $GITHUB_STEP_SUMMARY
-          echo '<p align=left>' >> $GITHUB_STEP_SUMMARY
-          jq -r --arg ts "$TIMESTAMP" '.[] | select(.Partnership_Status == "Gold Partner") |
-            "<a href='\''\(.Website)'\''><img src='\''\(.logo_url)?v=\($ts)'\'' width='\''105'\'' alt='\''\(.Account_Name)'\'' style=\"display:block; border:0; height:auto;\"></a>"' \
-            partners.json | tr '\n' ' ' >> $GITHUB_STEP_SUMMARY
-          echo '</p>' >> $GITHUB_STEP_SUMMARY
-          echo "" >> $GITHUB_STEP_SUMMARY
+          for level in Platinum Gold Silver; do
+            case $level in
+              Platinum) width=150 ;;
+              Gold)     width=105 ;;
+              Silver)   width=75 ;;
+            esac
+            echo "## ${level}" >> $GITHUB_STEP_SUMMARY
+            echo '<p align=left>' >> $GITHUB_STEP_SUMMARY
+            jq -r --arg ts "$TIMESTAMP" --arg status "${level} Partner" --arg w "$width" \
+              '.[] | select(.Partnership_Status == $status) |
+              "<a href='\''\(.Website)'\''><img src='\''\(.logo_url)?v=\($ts)'\'' width='\''\($w)'\'' alt='\''\(.Account_Name)'\'' style=\"display:block; border:0; height:auto;\"></a>"' \
+              partners.json | tr '\n' ' ' >> $GITHUB_STEP_SUMMARY
+            echo '</p>' >> $GITHUB_STEP_SUMMARY
+            echo "" >> $GITHUB_STEP_SUMMARY
+          done
 
-          # Silver Partners
-          echo "## Silver" >> $GITHUB_STEP_SUMMARY
-          echo '<p align=left>' >> $GITHUB_STEP_SUMMARY
-          jq -r --arg ts "$TIMESTAMP" '.[] | select(.Partnership_Status == "Silver Partner") |
-            "<a href='\''\(.Website)'\''><img src='\''\(.logo_url)?v=\($ts)'\'' width='\''75'\'' alt='\''\(.Account_Name)'\'' style=\"display:block; border:0; height:auto;\"></a>"' \
-            partners.json | tr '\n' ' ' >> $GITHUB_STEP_SUMMARY
-          echo '</p>' >> $GITHUB_STEP_SUMMARY
-          echo "" >> $GITHUB_STEP_SUMMARY
+          # Install Python dependencies for days_since_last_commit script
+          pip install --quiet requests
 
           # Create a temporary file for maintainers_with_avatars.json
           temp_file=$(mktemp)
@@ -104,24 +116,59 @@ jobs:
           # Flag for commas between records
           first=1
 
-          # Fetch the maintainers from Zoho Bigin
-          curl --silent --request GET \
-            --url 'https://www.zohoapis.eu/bigin/v2/Contacts/search?fields=Team,First_Name,Github,Maintaining,Your_core_competences&criteria=((Tag:equals:maintainer)and(Inactive:equals:false))' \
-            --header "Authorization: Zoho-oauthtoken $ACCESS_TOKEN" \
-          | jq -c '.data[] | {First_Name, Github, Team, Maintaining, Your_core_competences}' \
+          # Fetch the maintainers from Zoho Bigin (fail fast on error, same as partners fetch)
+          if ! maintainers_response=$(curl --silent --fail --request GET \
+            --url 'https://www.zohoapis.eu/bigin/v2/Contacts/search?fields=Team,First_Name,Last_Name,Github,Maintaining,Your_core_competences&criteria=((Tag:equals:maintainer)and(Inactive:equals:false))' \
+            --header "Authorization: Zoho-oauthtoken $ACCESS_TOKEN"); then
+            echo "::error::Failed to fetch maintainers from Zoho"
+            exit 1
+          fi
+
+          echo "$maintainers_response" \
+          | jq -c '.data[] | {First_Name, Last_Name, Github, Team, Maintaining, Your_core_competences}' \
           | while read -r row; do
               # Extract GitHub username from the URL
               github_url=$(echo "$row" | jq -r '.Github')
               username=$(basename "$github_url")
+              first_name=$(echo "$row" | jq -r '.First_Name // empty')
+              last_name=$(echo "$row" | jq -r '.Last_Name // empty')
 
-              # Assume GH_TOKEN is exported as env var or injected from GitHub Actions secrets
-              auth_header="Authorization: token $GH_TOKEN"
+              # Skip entries without a valid GitHub username
+              if [ -z "$username" ] || [ "$username" = "null" ] || [ "$username" = "." ]; then
+                echo "::warning::Skipping maintainer '${first_name} ${last_name}' - no GitHub profile set"
+                continue
+              fi
 
               # Fetch GitHub profile for avatar URL
-              avatar_url=$(curl -s -H "$auth_header" "https://api.github.com/users/$username" | jq -r '.avatar_url')
+              echo "Processing maintainer: $username (${first_name} ${last_name})"
+              avatar_url=$(curl -s -H "Authorization: token $GH_TOKEN" "https://api.github.com/users/$username" | jq -r '.avatar_url' || true)
 
-              # Enrich Zoho data with GitHub avatar
-              enriched=$(echo "$row" | jq --arg avatar "$avatar_url" '. + {Avatar: $avatar}')
+              # Fall back to empty string if avatar fetch failed
+              if [ "$avatar_url" = "null" ] || [ -z "$avatar_url" ]; then
+                echo "::warning::Could not fetch avatar for $username"
+                avatar_url=""
+              fi
+
+              # Get days since last activity in Armbian org (commits, issues)
+              echo "Fetching days since last activity for $username..."
+              days_since=$(python3 armbian.github.io/scripts/days_since_last_commit.py --days-only armbian "$username" "$GH_TOKEN" || true)
+              if [ "$days_since" = "ERROR" ] || [ -z "$days_since" ]; then
+                echo "::warning::Lookup failed for $username, skipping activity data"
+                days_since="-1"
+              elif [ "$days_since" = "-1" ]; then
+                echo "  -> $username: no activity found in org"
+              else
+                echo "  -> $username: ${days_since} days since last activity"
+              fi
+
+              # Enrich Zoho data with GitHub avatar, days since last activity, and Last_Name fallback
+              enriched=$(echo "$row" | jq --arg avatar "$avatar_url" --arg days "$days_since" '
+                . + {Avatar: $avatar, Days_since_last_activity: ($days | tonumber)}
+                | if (.First_Name == null or .First_Name == "") and (.Last_Name != null and .Last_Name != "")
+                  then .First_Name = .Last_Name
+                  else . end
+                | del(.Last_Name)
+              ')
 
               # Manage commas between JSON objects
               if [ $first -eq 1 ]; then
@@ -137,8 +184,16 @@ jobs:
           # Close the JSON array
           echo "]" >> "$temp_file"
 
-          # Format and save the final output to fixed.json
-          cat "$temp_file" | jq . > maintainers.json
+          # Format and save the final output
+          jq . "$temp_file" > maintainers.json
+
+          # Validate maintainers data
+          maintainer_count=$(jq 'length' maintainers.json)
+          if [ "$maintainer_count" -lt 1 ]; then
+            echo "::error::Maintainers JSON is empty, refusing to commit"
+            exit 1
+          fi
+          echo "Fetched $maintainer_count maintainers."
 
           # Clean up the temporary file
           rm "$temp_file"

scripts/days_since_last_commit.py

@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+import sys
+import time
+import requests
+from datetime import datetime, timezone
+
+API = "https://api.github.com"
+
+
+class LookupError(Exception):
+    pass
+
+
+def search_with_retry(url, headers, params, retries=5):
+    for attempt in range(retries):
+        try:
+            r = requests.get(url, headers=headers, params=params, timeout=30)
+        except requests.exceptions.RequestException as e:
+            raise LookupError(f"Request failed: {e}")
+
+        if r.status_code == 403:
+            retry_after = int(r.headers.get("Retry-After", 10 * (2 ** attempt)))
+            print(f"Rate limited, retrying in {retry_after}s (attempt {attempt + 1}/{retries})...", file=sys.stderr)
+            time.sleep(retry_after)
+            continue
+
+        if r.status_code == 422:
+            return None
+
+        if not r.ok:
+            raise LookupError(f"HTTP {r.status_code}")
+
+        return r.json()
+
+    raise LookupError("Rate limit retries exhausted")
+
+
+def get_latest_commit_date(org, user, headers):
+    data = search_with_retry(
+        f"{API}/search/commits", headers,
+        {"q": f"author:{user} org:{org}", "sort": "author-date", "order": "desc", "per_page": 1},
+    )
+    if not data or data.get("total_count", 0) == 0:
+        return None
+    date_str = data["items"][0]["commit"]["author"]["date"]
+    return datetime.fromisoformat(date_str.replace("Z", "+00:00"))
+
+
+def get_latest_issue_date(org, user, headers):
+    data = search_with_retry(
+        f"{API}/search/issues", headers,
+        {"q": f"author:{user} org:{org}", "sort": "created", "order": "desc", "per_page": 1},
+    )
+    if not data or data.get("total_count", 0) == 0:
+        return None
+    date_str = data["items"][0]["created_at"]
+    return datetime.fromisoformat(date_str.replace("Z", "+00:00"))
+
+
+def days_since_last_activity(org, user, token):
+    headers = {
+        "Accept": "application/vnd.github+json",
+        "Authorization": f"Bearer {token}",
+        "User-Agent": "org-activity-check",
+    }
+
+    dates = []
+    for fetch in (get_latest_commit_date, get_latest_issue_date):
+        dt = fetch(org, user, headers)  # raises LookupError on failure
+        if dt:
+            dates.append(dt)
+        time.sleep(1)
+
+    if not dates:
+        return None
+
+    latest = max(dates)
+    return (datetime.now(timezone.utc) - latest).days
+
+
+def main():
+    days_only = "--days-only" in sys.argv
+    args = [a for a in sys.argv[1:] if a != "--days-only"]
+
+    if len(args) != 3:
+        print("Usage: script.py [--days-only] ORG USER TOKEN")
+        sys.exit(1)
+
+    org, user, token = args[0], args[1], args[2]
+
+    try:
+        delta_days = days_since_last_activity(org, user, token)
+    except LookupError as e:
+        if days_only:
+            print("ERROR")
+        else:
+            print(f"Lookup failed for {user}: {e}")
+        sys.exit(1)
+
+    if delta_days is None:
+        if days_only:
+            print(-1)
+        else:
+            print(f"No activity found for {user} in org {org}")
+        sys.exit(0)
+
+    if days_only:
+        print(delta_days)
+    else:
+        print(f"Days since last activity: {delta_days}")
+
+
+if __name__ == "__main__":
+    main()