Add Phase 2 HTTPS prototype and runbook updates

.gitignore

@@ -3,6 +3,7 @@
 .codex/
 __pycache__/
 *.pyc
+tls/
 
 # Keep firmware SDK tree out of this workspace-tracking repo
 CR_SDK_CK-main/

PHASE5_RUNBOOK.md

@@ -2,6 +2,8 @@
 
 This runbook starts a minimal `k_server` + `k_proxy` prototype for session reuse testing.
 
+Last updated: 2026-04-25
+
 ## What This Prototype Covers
 
 - `k_proxy` creates short-lived sessions.
@@ -9,15 +11,26 @@ This runbook starts a minimal `k_server` + `k_proxy` prototype for session reuse
 - Valid sessions can repeatedly access a protected `k_server` counter endpoint without re-running card auth each request.
 - Session status and logout/invalidation paths are implemented.
 
+## Modes
+
+There are two useful ways to run this prototype:
+
+- Same-VM quickstart: `k_proxy` and `k_server` run on one VM for app-local testing.
+- Split-VM chain: `k_proxy` runs in `k_proxy`, `k_server` runs in `k_server`, and the Qubes forwarding layer must permit the chain.
+
 ## Start Services
 
-In `k_server` VM:
+### Same-VM quickstart
+
+This matches the code defaults and is useful for basic app behavior only.
+
+In the chosen VM:
 
 ```bash
 python3 /home/user/chromecard/k_server_app.py --host 127.0.0.1 --port 8780 --proxy-token dev-proxy-token
 ```
 
-In `k_proxy` VM:
+In the same VM:
 
 ```bash
 python3 /home/user/chromecard/k_proxy_app.py \
@@ -28,12 +41,64 @@ python3 /home/user/chromecard/k_proxy_app.py \
   --proxy-token dev-proxy-token
 ```
 
+### Split-VM chain
+
+This is the current Qubes target shape.
+
+In `k_server` VM:
+
+```bash
+python3 /home/user/chromecard/k_server_app.py \
+  --host 127.0.0.1 \
+  --port 8780 \
+  --proxy-token dev-proxy-token \
+  --tls-certfile /home/user/chromecard/tls/phase2/k_server.crt \
+  --tls-keyfile /home/user/chromecard/tls/phase2/k_server.key
+```
+
+In `k_proxy` VM:
+
+```bash
+qvm-connect-tcp 9780:k_server:8780
+```
+
+Notes:
+
+```bash
+python3 /home/user/chromecard/k_proxy_app.py \
+  --host 127.0.0.1 \
+  --port 8771 \
+  --session-ttl 300 \
+  --server-base-url https://127.0.0.1:9780 \
+  --server-ca-file /home/user/chromecard/tls/phase2/ca.crt \
+  --proxy-token dev-proxy-token \
+  --tls-certfile /home/user/chromecard/tls/phase2/k_proxy.crt \
+  --tls-keyfile /home/user/chromecard/tls/phase2/k_proxy.key
+```
+
+In `k_client` VM:
+
+```bash
+qvm-connect-tcp 9771:k_proxy:8771
+```
+
+Notes:
+
+- Current validated split-VM path is `k_client localhost:9771 -> k_proxy localhost:8771 -> k_proxy localhost:9780 forward -> k_server localhost:8780`.
+- Use `--cacert /home/user/chromecard/tls/phase2/ca.crt` for TLS verification in `curl`-based checks.
+- Raw VM-IP routing is not the validated path for the current prototype.
+
 ## Test Flow
 
+Use the proxy port that matches the mode you started:
+
+- Same-VM quickstart: `8770`
+- Split-VM chain: `9771` from `k_client`, `8771` inside `k_proxy`
+
 Create a session (runs auth gate once):
 
 ```bash
-curl -sS -X POST http://127.0.0.1:8770/session/login \
+curl -sS -X POST http://127.0.0.1:<proxy-port>/session/login \
   -H 'Content-Type: application/json' \
   -d '{"username":"alice"}'
 ```
@@ -47,30 +112,30 @@ TOKEN='<paste-token>'
 Check session:
 
 ```bash
-curl -sS -X POST http://127.0.0.1:8770/session/status \
+curl -sS -X POST http://127.0.0.1:<proxy-port>/session/status \
   -H "Authorization: Bearer $TOKEN"
 ```
 
 Call protected resource multiple times (should not require new login):
 
 ```bash
-curl -sS -X POST http://127.0.0.1:8770/resource/counter \
+curl -sS -X POST http://127.0.0.1:<proxy-port>/resource/counter \
   -H "Authorization: Bearer $TOKEN"
-curl -sS -X POST http://127.0.0.1:8770/resource/counter \
+curl -sS -X POST http://127.0.0.1:<proxy-port>/resource/counter \
   -H "Authorization: Bearer $TOKEN"
 ```
 
 Logout/invalidate:
 
 ```bash
-curl -sS -X POST http://127.0.0.1:8770/session/logout \
+curl -sS -X POST http://127.0.0.1:<proxy-port>/session/logout \
   -H "Authorization: Bearer $TOKEN"
 ```
 
 Re-check after logout (should fail with 401):
 
 ```bash
-curl -i -X POST http://127.0.0.1:8770/resource/counter \
+curl -i -X POST http://127.0.0.1:<proxy-port>/resource/counter \
   -H "Authorization: Bearer $TOKEN"
 ```
 
@@ -78,3 +143,4 @@ curl -i -X POST http://127.0.0.1:8770/resource/counter \
 
 - This uses card-presence probing, not a full WebAuthn assertion verification path.
 - Intended as a Phase 5 starter for session semantics and proxy/server behavior.
+- For the split-VM chain, the current blocker is not the Python prototype logic; it is refused `qubes.ConnectTCP` forwarding for the chain ports.

Setup.md

@@ -1,6 +1,6 @@
 # Setup
 
-Last updated: 2026-04-24
+Last updated: 2026-04-25
 
 This is a living setup/status file for the local ChromeCard workspace at `/home/user/chromecard`.
 Update this file whenever environment status or verified behavior changes.
@@ -239,6 +239,44 @@ Session note (2026-04-25, service restart):
   - `qrexec-client-vm k_proxy qubes.ConnectTCP+8771` -> `Request refused`
   - `qrexec-client-vm k_server qubes.ConnectTCP+8780` -> `Request refused`
 
+Session note (2026-04-25, markdown refresh):
+- Re-read the active workspace markdown files:
+  - `Setup.md`
+  - `Workplan.md`
+  - `PHASE5_RUNBOOK.md`
+- Corrected the Phase 5 runbook to distinguish the old same-VM quickstart from the current split-VM chain usage.
+- Current documented client-facing proxy port for split-VM tests is `8771`.
+- Current documented blocker remains unchanged:
+  - local service health inside `k_proxy` and `k_server` is good
+  - inter-VM forwarding via `qubes.ConnectTCP` is still refused
+
+Session note (2026-04-25, Phase 2 HTTPS bring-up):
+- Added direct TLS support to:
+  - `/home/user/chromecard/k_proxy_app.py`
+  - `/home/user/chromecard/k_server_app.py`
+- Added local certificate generator:
+  - `/home/user/chromecard/generate_phase2_certs.py`
+- Generated local CA and service certs at:
+  - `/home/user/chromecard/tls/phase2/ca.crt`
+  - `/home/user/chromecard/tls/phase2/k_proxy.crt`
+  - `/home/user/chromecard/tls/phase2/k_server.crt`
+- Certificate generation was corrected to include subject key identifier and authority key identifier so Python TLS verification succeeds.
+- Current validated HTTPS shape is Qubes-localhost forwarding, not raw VM-IP routing:
+  - in `k_client`: `qvm-connect-tcp 9771:k_proxy:8771`
+  - in `k_proxy`: `qvm-connect-tcp 9780:k_server:8780`
+  - `k_proxy` listens on `https://127.0.0.1:8771`
+  - `k_server` listens on `https://127.0.0.1:8780`
+  - `k_proxy` upstream is `https://127.0.0.1:9780`
+- Verified HTTPS checks:
+  - `k_client -> k_proxy` `/health` over TLS succeeds with `--cacert /home/user/chromecard/tls/phase2/ca.crt`
+  - `k_proxy -> k_server` `/health` and `/resource/counter` over TLS succeed through the `9780` forwarder
+  - end-to-end `k_client -> k_proxy -> k_server` login + session reuse succeeded over HTTPS
+- End-to-end verified results:
+  - login returned `ok=true` for `alice`
+  - first protected counter call returned value `1`
+  - second protected counter call returned value `2`
+  - session status remained valid after reuse
+
 Session note (2026-04-25, in-VM forwarding test):
 - Tested the intended in-VM forwarding path with `qvm-connect-tcp` instead of host-side `qrexec-client-vm`.
 - Forwarders start and bind locally:

Workplan.md

@@ -1,6 +1,6 @@
 # Workplan
 
-Last updated: 2026-04-24
+Last updated: 2026-04-25
 
 This is the execution plan for making ChromeCard FIDO2 development and validation reproducible on this machine.
 
@@ -66,6 +66,16 @@ Status (2026-04-24, remote diagnostics):
   - `k_proxy (10.137.0.12) -> k_server (10.137.0.13:8780)`: upstream timeout.
 - Local service health inside each VM is good, so failure is inter-VM reachability, not local process startup.
 
+Status (2026-04-25, after restart and service recovery):
+- Refined blocker: this is currently a qrexec/`qubes.ConnectTCP` refusal problem, not an app-local listener problem.
+- Current evidence:
+  - `k_proxy` local `/health` is up on `127.0.0.1:8771`
+  - `k_server` local `/health` is up on `127.0.0.1:8780`
+  - `qrexec-client-vm k_proxy qubes.ConnectTCP+8771` -> `Request refused`
+  - `qrexec-client-vm k_server qubes.ConnectTCP+8780` -> `Request refused`
+- Immediate next action for Phase 1:
+  - verify and fix the dom0 policy/mechanism that should permit `qubes.ConnectTCP` forwarding for the chain ports
+
 ## Phase 2: TLS Certificates and Service Endpoints
 
 1. Certificate model.
@@ -84,6 +94,19 @@ Exit criteria:
 - Mutual TLS trust decisions are documented and tested.
 - HTTPS calls succeed on both links with expected cert validation.
 
+Status (2026-04-25):
+- Implemented HTTPS listeners in both prototype services.
+- Added local CA + service certificate generation in `generate_phase2_certs.py`.
+- Verified the working Qubes path is localhost forwarding plus TLS:
+  - `k_client` local `9771` forwards to `k_proxy:8771`
+  - `k_proxy` local `9780` forwards to `k_server:8780`
+- Verified cert validation on both hops using the generated CA.
+- Verified end-to-end HTTPS flow:
+  - `k_client -> k_proxy` login over TLS
+  - `k_proxy -> k_server` protected counter call over TLS
+  - session reuse still works across repeated protected requests
+- Phase 2 is now effectively complete for the current prototype shape.
+
 ## Phase 2.5: Define State Ownership and Concurrency Model
 
 1. State ownership.
@@ -100,6 +123,13 @@ Exit criteria:
 Exit criteria:
 - Architecture clearly documents state authority and race-free update rules.
 
+Next action (2026-04-25):
+- Move into Phase 2.5 and make the current prototype decisions explicit:
+  - authority for session state remains `k_proxy`
+  - `k_server` remains authority for the protected counter/resource state
+  - localhost Qubes forwarders are part of the active runtime model for the two TLS hops
+  - define concurrency assumptions and limits around session store, forwarders, and counter access
+
 ## Phase 3: Recover Basic Device Visibility on `k_proxy` (Blocking)
 
 1. Verify physical + USB enumeration path.
@@ -173,6 +203,13 @@ Status (2026-04-24):
   - proxy forwarding from `k_proxy` to `k_server` using a shared upstream token
 - Current auth gate for session creation is card-presence probe (`fido2_probe.py --json`), pending upgrade to full assertion verification path.
 
+Status (2026-04-25):
+- Prototype services were re-started successfully after VM restart.
+- Current split-VM test shape is:
+  - `k_proxy` listening on `127.0.0.1:8771`
+  - `k_server` listening on `127.0.0.1:8780`
+- Phase 5 application logic is runnable locally inside each VM, but end-to-end validation is still blocked by Phase 1 qrexec forwarding refusal.
+
 ## Phase 5.5: Implement Dummy Resource + Access Policy on `k_server`
 
 1. Protected dummy resource.

generate_phase2_certs.py

@@ -0,0 +1,157 @@
+#!/usr/bin/env python3
+"""
+Generate a small local CA plus leaf certificates for Phase 2 HTTPS testing.
+"""
+
+from __future__ import annotations
+
+import argparse
+import ipaddress
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+
+from cryptography import x509
+from cryptography.hazmat.primitives import hashes, serialization
+from cryptography.hazmat.primitives.asymmetric import rsa
+from cryptography.x509.oid import ExtendedKeyUsageOID, NameOID
+
+
+def build_name(common_name: str) -> x509.Name:
+    return x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, common_name)])
+
+
+def new_private_key() -> rsa.RSAPrivateKey:
+    return rsa.generate_private_key(public_exponent=65537, key_size=2048)
+
+
+def write_private_key(path: Path, key: rsa.RSAPrivateKey) -> None:
+    path.write_bytes(
+        key.private_bytes(
+            encoding=serialization.Encoding.PEM,
+            format=serialization.PrivateFormat.TraditionalOpenSSL,
+            encryption_algorithm=serialization.NoEncryption(),
+        )
+    )
+
+
+def write_cert(path: Path, cert: x509.Certificate) -> None:
+    path.write_bytes(cert.public_bytes(serialization.Encoding.PEM))
+
+
+def parse_sans(names: list[str]) -> list[x509.GeneralName]:
+    sans: list[x509.GeneralName] = []
+    seen = set()
+    for value in names:
+        if value in seen:
+            continue
+        seen.add(value)
+        try:
+            sans.append(x509.IPAddress(ipaddress.ip_address(value)))
+        except ValueError:
+            sans.append(x509.DNSName(value))
+    return sans
+
+
+def issue_ca(common_name: str, valid_days: int) -> tuple[rsa.RSAPrivateKey, x509.Certificate]:
+    now = datetime.now(timezone.utc)
+    key = new_private_key()
+    subject = issuer = build_name(common_name)
+    cert = (
+        x509.CertificateBuilder()
+        .subject_name(subject)
+        .issuer_name(issuer)
+        .public_key(key.public_key())
+        .serial_number(x509.random_serial_number())
+        .not_valid_before(now - timedelta(minutes=5))
+        .not_valid_after(now + timedelta(days=valid_days))
+        .add_extension(x509.BasicConstraints(ca=True, path_length=None), critical=True)
+        .add_extension(x509.SubjectKeyIdentifier.from_public_key(key.public_key()), critical=False)
+        .add_extension(x509.AuthorityKeyIdentifier.from_issuer_public_key(key.public_key()), critical=False)
+        .add_extension(x509.KeyUsage(digital_signature=True, key_encipherment=False, key_cert_sign=True, crl_sign=True, content_commitment=False, data_encipherment=False, key_agreement=False, encipher_only=False, decipher_only=False), critical=True)
+        .sign(key, hashes.SHA256())
+    )
+    return key, cert
+
+
+def issue_leaf(
+    ca_key: rsa.RSAPrivateKey,
+    ca_cert: x509.Certificate,
+    common_name: str,
+    san_values: list[str],
+    valid_days: int,
+) -> tuple[rsa.RSAPrivateKey, x509.Certificate]:
+    now = datetime.now(timezone.utc)
+    key = new_private_key()
+    cert = (
+        x509.CertificateBuilder()
+        .subject_name(build_name(common_name))
+        .issuer_name(ca_cert.subject)
+        .public_key(key.public_key())
+        .serial_number(x509.random_serial_number())
+        .not_valid_before(now - timedelta(minutes=5))
+        .not_valid_after(now + timedelta(days=valid_days))
+        .add_extension(x509.BasicConstraints(ca=False, path_length=None), critical=True)
+        .add_extension(x509.SubjectAlternativeName(parse_sans(san_values)), critical=False)
+        .add_extension(x509.SubjectKeyIdentifier.from_public_key(key.public_key()), critical=False)
+        .add_extension(x509.AuthorityKeyIdentifier.from_issuer_public_key(ca_key.public_key()), critical=False)
+        .add_extension(x509.ExtendedKeyUsage([ExtendedKeyUsageOID.SERVER_AUTH]), critical=False)
+        .add_extension(x509.KeyUsage(digital_signature=True, key_encipherment=True, key_cert_sign=False, crl_sign=False, content_commitment=False, data_encipherment=False, key_agreement=False, encipher_only=False, decipher_only=False), critical=True)
+        .sign(ca_key, hashes.SHA256())
+    )
+    return key, cert
+
+
+def emit_leaf_bundle(
+    out_dir: Path,
+    leaf_name: str,
+    ca_key: rsa.RSAPrivateKey,
+    ca_cert: x509.Certificate,
+    san_values: list[str],
+    valid_days: int,
+) -> None:
+    key, cert = issue_leaf(ca_key, ca_cert, leaf_name, san_values, valid_days)
+    write_private_key(out_dir / f"{leaf_name}.key", key)
+    write_cert(out_dir / f"{leaf_name}.crt", cert)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="Generate local CA and Phase 2 service certificates")
+    parser.add_argument("--out-dir", default="tls/phase2")
+    parser.add_argument("--valid-days", type=int, default=30)
+    parser.add_argument("--ca-common-name", default="ChromeCard Phase2 Local CA")
+    parser.add_argument(
+        "--proxy-san",
+        action="append",
+        default=[],
+        help="Extra SAN for k_proxy certificate; may be repeated",
+    )
+    parser.add_argument(
+        "--server-san",
+        action="append",
+        default=[],
+        help="Extra SAN for k_server certificate; may be repeated",
+    )
+    return parser.parse_args()
+
+
+def main() -> int:
+    args = parse_args()
+    out_dir = Path(args.out_dir)
+    out_dir.mkdir(parents=True, exist_ok=True)
+
+    ca_key, ca_cert = issue_ca(args.ca_common_name, args.valid_days)
+    write_private_key(out_dir / "ca.key", ca_key)
+    write_cert(out_dir / "ca.crt", ca_cert)
+
+    proxy_sans = ["localhost", "127.0.0.1", "k_proxy", *args.proxy_san]
+    server_sans = ["localhost", "127.0.0.1", "k_server", *args.server_san]
+
+    emit_leaf_bundle(out_dir, "k_proxy", ca_key, ca_cert, proxy_sans, args.valid_days)
+    emit_leaf_bundle(out_dir, "k_server", ca_key, ca_cert, server_sans, args.valid_days)
+
+    print(f"Generated CA and leaf certificates in {out_dir}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

k_proxy_app.py

@@ -17,6 +17,7 @@ from __future__ import annotations
 import argparse
 import json
 import secrets
+import ssl
 import subprocess
 import threading
 import time
@@ -40,11 +41,13 @@ class ProxyState:
         session_ttl_s: int,
         auth_command: str,
         server_base_url: str,
+        server_ca_file: str | None,
         proxy_token: str,
     ):
         self.session_ttl_s = session_ttl_s
         self.auth_command = auth_command
         self.server_base_url = server_base_url.rstrip("/")
+        self.server_ca_file = server_ca_file
         self.proxy_token = proxy_token
         self.lock = threading.Lock()
         self.sessions: dict[str, Session] = {}
@@ -108,8 +111,11 @@ class ProxyState:
         req.add_header("X-Proxy-Token", self.proxy_token)
         req.add_header("Content-Type", "application/json")
         body = b"{}"
+        ssl_context = None
+        if self.server_base_url.startswith("https://"):
+            ssl_context = ssl.create_default_context(cafile=self.server_ca_file)
         try:
-            with urlopen(req, data=body, timeout=5) as resp:
+            with urlopen(req, data=body, timeout=5, context=ssl_context) as resp:
                 data = json.loads(resp.read().decode("utf-8"))
                 return resp.status, data
         except HTTPError as exc:
@@ -268,6 +274,8 @@ def parse_args() -> argparse.Namespace:
     parser = argparse.ArgumentParser(description="Run k_proxy session gateway")
     parser.add_argument("--host", default="127.0.0.1")
     parser.add_argument("--port", type=int, default=8770)
+    parser.add_argument("--tls-certfile", help="PEM certificate chain for HTTPS listener")
+    parser.add_argument("--tls-keyfile", help="PEM private key for HTTPS listener")
     parser.add_argument("--session-ttl", type=int, default=300, help="Session TTL in seconds")
     parser.add_argument(
         "--auth-command",
@@ -279,6 +287,10 @@ def parse_args() -> argparse.Namespace:
         default="http://127.0.0.1:8780",
         help="Base URL for k_server",
     )
+    parser.add_argument(
+        "--server-ca-file",
+        help="CA certificate used to verify HTTPS certificate presented by k_server",
+    )
     parser.add_argument(
         "--proxy-token",
         default="dev-proxy-token",
@@ -289,15 +301,28 @@ def parse_args() -> argparse.Namespace:
 
 def main() -> int:
     args = parse_args()
+    if bool(args.tls_certfile) != bool(args.tls_keyfile):
+        raise SystemExit("Both --tls-certfile and --tls-keyfile are required to enable HTTPS")
+    if args.server_base_url.startswith("https://") and not args.server_ca_file:
+        raise SystemExit("--server-ca-file is required when --server-base-url uses https")
+
     state = ProxyState(
         session_ttl_s=args.session_ttl,
         auth_command=args.auth_command,
         server_base_url=args.server_base_url,
+        server_ca_file=args.server_ca_file,
         proxy_token=args.proxy_token,
     )
     Handler.state = state
     server = ThreadingHTTPServer((args.host, args.port), Handler)
-    print(f"k_proxy listening on http://{args.host}:{args.port}")
+    scheme = "http"
+    if args.tls_certfile:
+        context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
+        context.load_cert_chain(certfile=args.tls_certfile, keyfile=args.tls_keyfile)
+        server.socket = context.wrap_socket(server.socket, server_side=True)
+        scheme = "https"
+
+    print(f"k_proxy listening on {scheme}://{args.host}:{args.port}")
     server.serve_forever()
     return 0
 

k_server_app.py

@@ -12,6 +12,7 @@ from __future__ import annotations
 
 import argparse
 import json
+import ssl
 import threading
 import time
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
@@ -84,6 +85,8 @@ def parse_args() -> argparse.Namespace:
     parser = argparse.ArgumentParser(description="Run k_server counter service")
     parser.add_argument("--host", default="127.0.0.1")
     parser.add_argument("--port", type=int, default=8780)
+    parser.add_argument("--tls-certfile", help="PEM certificate chain for HTTPS listener")
+    parser.add_argument("--tls-keyfile", help="PEM private key for HTTPS listener")
     parser.add_argument(
         "--proxy-token",
         default="dev-proxy-token",
@@ -94,10 +97,20 @@ def parse_args() -> argparse.Namespace:
 
 def main() -> int:
     args = parse_args()
+    if bool(args.tls_certfile) != bool(args.tls_keyfile):
+        raise SystemExit("Both --tls-certfile and --tls-keyfile are required to enable HTTPS")
+
     state = ServerState(proxy_token=args.proxy_token)
     Handler.state = state
     server = ThreadingHTTPServer((args.host, args.port), Handler)
-    print(f"k_server listening on http://{args.host}:{args.port}")
+    scheme = "http"
+    if args.tls_certfile:
+        context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
+        context.load_cert_chain(certfile=args.tls_certfile, keyfile=args.tls_keyfile)
+        server.socket = context.wrap_socket(server.socket, server_side=True)
+        scheme = "https"
+
+    print(f"k_server listening on {scheme}://{args.host}:{args.port}")
     server.serve_forever()
     return 0