NeahNew/PROJECT_DEEP_ANALYSIS.md

Neah Project - Deep Technical Analysis

Executive Summary

This document provides a comprehensive analysis of the Neah project architecture, focusing on:

• Update Services & Refresh Management
• Widgets Architecture
• Notifications System
• Authentication & Token Refresh
• Performance & Memory Management
• API Routes Tracing

---

1. Update Services & Refresh Management

1.1 Unified Refresh Manager (lib/services/refresh-manager.ts)

Architecture:

• Singleton Pattern: Single instance manages all refresh operations
• Resource-Based: Each refreshable resource has its own configuration
• Deduplication: Prevents duplicate refresh requests
• Interval Management: Centralized interval control

Refreshable Resources:

type RefreshableResource = 
  | 'notifications'
  | 'notifications-count'
  | 'calendar'
  | 'news'
  | 'email'
  | 'parole'
  | 'duties'
  | 'navbar-time';

Key Features:

1. Request Deduplication

   • Minimum 1 second between refreshes for same resource
   • Tracks pending requests to prevent duplicates
   • Uses pendingRequests Map with promise tracking

2. Interval Management

   • Each resource can have different refresh intervals
   • Automatic cleanup on unregister
   • Pause/Resume functionality for all resources

3. Error Handling

   • Errors don't update lastRefresh timestamp (allows retry)
   • Comprehensive logging for debugging
   • Graceful degradation on failures

Memory Impact:

• Low: Uses Maps for efficient lookups
• Cleanup: Proper cleanup on component unmount
• Potential Issue: If components don't unregister, intervals persist

Performance Considerations:

• ✅ Deduplication prevents unnecessary API calls
• ✅ Minimum 1s throttle prevents excessive refreshes
• ⚠️ Multiple resources = multiple intervals (but necessary)
• ⚠️ No priority-based scheduling (all resources treated equally)

1.2 Unified Refresh Hook (hooks/use-unified-refresh.ts)

Purpose: React hook wrapper for RefreshManager

Key Features:

• Automatic registration/unregistration on mount/unmount
• Session-aware (only active when authenticated)
• Callback ref pattern to avoid stale closures
• Manual refresh trigger with force option

Usage Pattern:

const { refresh, isActive } = useUnifiedRefresh({
  resource: 'calendar',
  interval: 300000, // 5 minutes
  enabled: status === 'authenticated',
  onRefresh: fetchEvents,
  priority: 'low',
});

Memory Leak Prevention:

• ✅ Cleanup in useEffect return
• ✅ isMountedRef prevents state updates after unmount
• ✅ Automatic unregister on unmount

---

2. Widgets Architecture

2.1 Widget Components Overview

Main Dashboard Widgets (app/page.tsx):

1. QuoteCard - Inspirational quotes
2. Calendar - Upcoming events (7 events)
3. News - News articles (100 limit)
4. Duties - Leantime tasks (7 tasks)
5. Email - Email preview (5 emails)
6. Parole - RocketChat messages

2.2 Widget Refresh Patterns

Current Implementation Issues:

1. Calendar Widget (components/calendar.tsx)

   • ❌ No unified refresh integration
   • ❌ Manual refresh only via button
   • ❌ Fetches on mount only
   • ⚠️ Uses ?refresh=true parameter (bypasses cache)

2. News Widget (components/news.tsx)

   • ❌ No unified refresh integration
   • ✅ Manual refresh button
   • ✅ Fetches on authentication
   • ⚠️ Uses ?refresh=true parameter

3. Email Widget (components/email.tsx)

   • ❌ No unified refresh integration
   • ✅ Manual refresh button
   • ⚠️ Fetches on mount only
   • ⚠️ Uses ?refresh=true parameter

4. Parole Widget (components/parole.tsx)

   • ❌ No unified refresh integration
   • ⚠️ Custom polling: setInterval(() => fetchMessages(), 30000) (30s)
   • ⚠️ Memory Leak Risk: Interval not cleared if component unmounts during fetch
   • ✅ Manual refresh button

5. Duties Widget (components/flow.tsx)

   • ❌ No unified refresh integration
   • ❌ Fetches on mount only
   • ⚠️ Uses ?refresh=true parameter

2.3 Widget Memory & Performance Issues

Critical Issues:

1. Multiple Polling Mechanisms

   • Parole widget uses setInterval (30s)
   • No coordination with RefreshManager
   • Risk of memory leaks if cleanup fails

2. Cache Bypassing

   • Most widgets use ?refresh=true
   • Bypasses Redis cache
   • Increases server load

3. No Unified Refresh

   • Widgets don't use useUnifiedRefresh hook
   • Inconsistent refresh patterns
   • Hard to manage globally

4. State Management

   • Each widget manages its own state
   • No shared state/cache
   • Potential duplicate API calls

Recommendations:

• ✅ Migrate all widgets to use useUnifiedRefresh
• ✅ Remove custom setInterval implementations
• ✅ Use cache-first strategy (remove ?refresh=true by default)
• ✅ Implement widget-level error boundaries

---

3. Notifications System

3.1 Architecture Overview

Service Pattern: Singleton with adapter pattern

Location: lib/services/notifications/notification-service.ts

Adapters:

• LeantimeAdapter (implemented)
• NextcloudAdapter (planned)
• GiteaAdapter (planned)
• DolibarrAdapter (planned)
• MoodleAdapter (planned)

3.2 Caching Strategy

Redis Cache Keys:

NOTIFICATION_COUNT_CACHE_KEY = `notifications:count:${userId}`
NOTIFICATIONS_LIST_CACHE_KEY = `notifications:list:${userId}:${page}:${limit}`

Cache TTL:

• Count cache: 30 seconds
• List cache: 30 seconds
• Refresh lock: 30 seconds

Cache Invalidation:

• On markAsRead: Invalidates all user caches
• Uses Redis SCAN for pattern matching
• Prevents blocking operations

3.3 Refresh Management

Integration with RefreshManager:

• ✅ Uses unified refresh system
• ✅ Registered as 'notifications' and 'notifications-count'
• ✅ 30-second refresh interval (aligned with cache TTL)

Hook Usage (hooks/use-notifications.ts):

• Request deduplication (2-second window)
• Automatic refresh on mount
• Manual refresh capability
• Error handling with retry

3.4 Performance Characteristics

Strengths:

• ✅ Redis caching reduces database load
• ✅ Adapter pattern allows easy extension
• ✅ Parallel fetching from multiple adapters
• ✅ Request deduplication prevents duplicate calls

Potential Issues:

• ⚠️ SCAN operations can be slow with many keys
• ⚠️ No pagination limits on adapter results
• ⚠️ All adapters fetched in parallel (could be optimized)

Memory Impact:

• Low: Cached data in Redis, not memory
• Medium: Notification objects in React state
• Low: Adapter instances are singletons

---

4. Authentication & Token Refresh

4.1 Keycloak Integration

Provider: NextAuth with KeycloakProvider

Location: app/api/auth/options.ts

4.2 Token Refresh Flow

JWT Callback Logic:

1. Initial Sign-In:

   • Stores access token, refresh token, ID token
   • Extracts roles from access token
   • Sets expiration timestamp

2. Subsequent Requests:

   • Checks if token is expired
   • If expired, calls refreshAccessToken()
   • Updates token with new values

Refresh Function (refreshAccessToken):

async function refreshAccessToken(token: ExtendedJWT) {
  // Calls Keycloak token endpoint
  // Handles various error scenarios:
  // - SessionNotActive (user logged out)
  // - RefreshTokenExpired (inactivity)
  // - InvalidGrant (session invalidated)
}

Error Handling:

• ✅ Detects session invalidation
• ✅ Handles refresh token expiration
• ✅ Clears tokens on critical errors
• ✅ Returns null session to trigger re-auth

4.3 Session Management

Session Configuration:

• Strategy: JWT (stateless)
• Max Age: 4 hours (14,400 seconds)
• Automatic refresh on activity

Cookie Configuration:

• HttpOnly: true
• SameSite: 'lax'
• Secure: Based on NEXTAUTH_URL

4.4 Email OAuth Token Refresh

Service: lib/services/token-refresh.ts

Purpose: Refresh Microsoft OAuth tokens for email access

Flow:

1. Check Redis cache for credentials
2. If cache miss, check Prisma database
3. Validate token expiration (5-minute buffer)
4. Refresh if needed via Microsoft OAuth
5. Update both Redis and Prisma

Dual Storage:

• Redis: Fast access, 24-hour TTL
• Prisma: Persistent storage, survives Redis restarts

Memory Impact:

• Low: Credentials stored in Redis/DB, not memory
• Medium: Token refresh operations are async
• Low: No memory leaks (proper cleanup)

4.5 Performance Considerations

Token Refresh Frequency:

• Keycloak: On every request if expired
• Email OAuth: Only when expired (5-min buffer)

Optimization Opportunities:

• ⚠️ Token refresh happens synchronously in JWT callback
• ⚠️ Could implement background refresh
• ✅ Caching reduces refresh frequency

---

5. Performance & Memory Management

5.1 Next.js Configuration

Build Configuration (next.config.mjs):

experimental: {
  webpackBuildWorker: true,
  parallelServerBuildTraces: true,
  parallelServerCompiles: true,
}

Memory Impact:

• ✅ Parallel builds reduce build time
• ⚠️ Multiple workers increase memory during build
• ✅ Production builds are optimized

5.2 Redis Connection Management

Singleton Pattern (lib/redis.ts):

• Single Redis client instance
• Connection pooling
• Automatic reconnection with retry strategy

Memory Impact:

• Low: Single connection per process
• Medium: Connection pool (if configured)
• Low: Proper cleanup on disconnect

Connection Strategy:

• Max reconnect attempts: 5
• Exponential backoff
• Connection timeout: 10 seconds
• Keep-alive: 10 seconds

5.3 Caching Strategy

Redis Cache TTLs:

CREDENTIALS: 24 hours
SESSION: 4 hours
EMAIL_LIST: 5 minutes
EMAIL_CONTENT: 15 minutes
CALENDAR: 10 minutes
NEWS: 15 minutes
TASKS: 10 minutes
MESSAGES: 2 minutes
NOTIFICATIONS: 30 seconds

Memory Impact:

• Low: Data in Redis, not application memory
• Medium: Large cache can consume Redis memory
• Low: TTL ensures automatic cleanup

5.4 Component Memory Management

Potential Memory Leaks:

1. Parole Widget (components/parole.tsx):

   // ⚠️ RISK: Interval might not clear if component unmounts during fetch
   useEffect(() => {
     if (status === 'authenticated') {
       fetchMessages();
       const interval = setInterval(() => fetchMessages(), 30000);
       return () => clearInterval(interval); // ✅ Good, but...
     }
   }, [status]);
   

   Issue: If fetchMessages() is async and component unmounts, state updates may occur

2. Widget State:

   • Each widget maintains its own state
   • No cleanup on unmount for pending requests
   • Potential memory leaks with large data arrays

3. Event Listeners:

   • No evidence of unregistered event listeners
   • ✅ React handles most cleanup automatically

5.5 API Route Performance

Common Patterns:

1. Session Validation:

   const session = await getServerSession(authOptions);
   

   • Called on every request
   • JWT validation overhead
   • Could be optimized with middleware

2. Database Queries:

   • Prisma ORM adds overhead
   • No query optimization visible
   • Connection pooling handled by Prisma

3. Redis Operations:

   • Most routes check cache first
   • SCAN operations for pattern matching
   • Could be optimized with better key patterns

5.6 Memory Optimization Recommendations

High Priority:

1. ✅ Fix Parole widget interval cleanup
2. ✅ Migrate widgets to unified refresh
3. ✅ Implement request cancellation for unmounted components
4. ✅ Add error boundaries to prevent memory leaks

Medium Priority:

1. ⚠️ Implement API route middleware for auth
2. ⚠️ Optimize Redis SCAN operations
3. ⚠️ Add request timeout handling
4. ⚠️ Implement connection pooling for external APIs

Low Priority:

1. ⚠️ Consider React Query for state management
2. ⚠️ Implement virtual scrolling for large lists
3. ⚠️ Add memory profiling tools

---

6. API Routes Tracing

6.1 Logging Infrastructure

Logger (lib/logger.ts):

• Environment-aware (silent in production for debug/info)
• Always logs errors
• Simple console-based logging

Limitations:

• ❌ No structured logging (JSON)
• ❌ No log levels in production
• ❌ No centralized log aggregation
• ❌ No request tracing IDs

6.2 Current Logging Patterns

API Routes:

• 343 console.log/error/warn calls across 68 files
• Inconsistent logging patterns
• Some routes have detailed logging, others minimal

Examples:

1. Good Logging (app/api/missions/mission-created/route.ts):

   logger.debug('Mission Created Webhook Received');
   logger.debug('Received mission-created data', { ... });
   

2. Inconsistent Logging (app/api/courrier/route.ts):

   console.log(`[API] Received request with: ...`);
   // Mix of console.log and logger
   

6.3 API Route Categories

Authentication Routes:

• /api/auth/[...nextauth] - NextAuth handler
• /api/auth/refresh-keycloak-session - Session refresh
• /api/auth/debug-keycloak - Debug endpoint

Email Routes (Courrier):

• /api/courrier - Email list
• /api/courrier/emails - Email list (alternative)
• /api/courrier/[id] - Single email
• /api/courrier/refresh - Token refresh
• /api/courrier/session - IMAP session
• /api/courrier/account - Account management

Calendar Routes:

• /api/calendars - Calendar list
• /api/calendars/[id] - Single calendar
• /api/calendars/[id]/events - Calendar events
• /api/events - Event CRUD

Notification Routes:

• /api/notifications - Notification list
• /api/notifications/count - Notification count
• /api/notifications/[id]/read - Mark as read
• /api/notifications/read-all - Mark all as read

Mission Routes:

• /api/missions - Mission list
• /api/missions/[missionId] - Single mission
• /api/missions/upload - File upload
• /api/missions/mission-created - Webhook handler

6.4 Tracing Recommendations

Immediate Improvements:

1. Request ID Tracking:

   // Add to middleware or API route wrapper
   const requestId = crypto.randomUUID();
   logger.info('Request started', { requestId, path, method });
   

2. Structured Logging:

   logger.info('API Request', {
     requestId,
     method,
     path,
     userId,
     duration: Date.now() - startTime,
   });
   

3. Error Tracking:

   logger.error('API Error', {
     requestId,
     error: error.message,
     stack: error.stack,
     path,
     userId,
   });
   

4. Performance Monitoring:

   const startTime = Date.now();
   // ... route logic
   logger.debug('API Response', {
     requestId,
     duration: Date.now() - startTime,
     statusCode,
   });
   

Advanced Tracing:

1. OpenTelemetry Integration:

   • Distributed tracing
   • Performance metrics
   • Error tracking

2. APM Tools:

   • New Relic
   • Datadog
   • Sentry

3. Custom Middleware:

   // app/api/middleware.ts
   export function withTracing(handler: Function) {
     return async (req: Request, res: Response) => {
       const requestId = crypto.randomUUID();
       const startTime = Date.now();
   
       try {
         const result = await handler(req, res);
         logger.info('Request completed', {
           requestId,
           duration: Date.now() - startTime,
         });
         return result;
       } catch (error) {
         logger.error('Request failed', {
           requestId,
           error,
           duration: Date.now() - startTime,
         });
         throw error;
       }
     };
   }
   

6.5 API Route Performance Metrics

Current State:

• ❌ No performance metrics collected
• ❌ No request duration tracking
• ❌ No error rate monitoring
• ❌ No cache hit/miss tracking

Recommended Metrics:

1. Request duration (p50, p95, p99)
2. Error rate by route
3. Cache hit/miss ratio
4. Database query count
5. Redis operation count
6. External API call duration

---

7. Critical Issues & Recommendations

7.1 Critical Issues

1. Memory Leak Risk - Parole Widget

   • Custom setInterval without proper cleanup
   • Fix: Migrate to useUnifiedRefresh

2. Inconsistent Refresh Patterns

   • Widgets don't use unified refresh system
   • Fix: Migrate all widgets to useUnifiedRefresh

3. Cache Bypassing

   • Widgets use ?refresh=true by default
   • Fix: Use cache-first strategy

4. No Request Tracing

   • Difficult to debug production issues
   • Fix: Implement request ID tracking

5. No Performance Monitoring

   • No visibility into slow routes
   • Fix: Add performance metrics

7.2 High Priority Recommendations

1. ✅ Migrate all widgets to unified refresh system
2. ✅ Fix Parole widget interval cleanup
3. ✅ Implement request ID tracking
4. ✅ Add performance metrics
5. ✅ Standardize logging patterns

7.3 Medium Priority Recommendations

1. ⚠️ Implement API route middleware
2. ⚠️ Optimize Redis SCAN operations
3. ⚠️ Add error boundaries
4. ⚠️ Implement request cancellation
5. ⚠️ Add structured logging

7.4 Low Priority Recommendations

1. ⚠️ Consider React Query
2. ⚠️ Implement virtual scrolling
3. ⚠️ Add memory profiling
4. ⚠️ Consider OpenTelemetry
5. ⚠️ Add APM tooling

---

8. Architecture Strengths

8.1 Well-Designed Components

1. Unified Refresh Manager

   • Excellent abstraction
   • Proper deduplication
   • Clean API

2. Notification Service

   • Adapter pattern allows extension
   • Good caching strategy
   • Proper error handling

3. Redis Integration

   • Comprehensive caching
   • Proper TTL management
   • Good key naming conventions

4. Token Refresh

   • Dual storage (Redis + Prisma)
   • Proper error handling
   • Automatic refresh

8.2 Code Quality

• ✅ TypeScript throughout
• ✅ Consistent component structure
• ✅ Proper error handling in most places
• ✅ Good separation of concerns

---

9. Conclusion

The Neah project demonstrates a well-architected Next.js application with several sophisticated systems:

Strengths:

• Unified refresh management system
• Comprehensive caching strategy
• Robust authentication flow
• Extensible notification system

Areas for Improvement:

• Widget refresh consistency
• Memory leak prevention
• API route tracing
• Performance monitoring

Overall Assessment: The codebase is production-ready but would benefit from the recommended improvements, particularly around widget refresh management and observability.

---

Appendix: File Reference Map

Core Services

• lib/services/refresh-manager.ts - Unified refresh management
• lib/services/notifications/notification-service.ts - Notification system
• lib/services/token-refresh.ts - Email OAuth token refresh
• lib/redis.ts - Redis caching utilities
• lib/logger.ts - Logging utility

Hooks

• hooks/use-unified-refresh.ts - Unified refresh hook
• hooks/use-notifications.ts - Notification hook

Widgets

• components/calendar.tsx - Calendar widget
• components/news.tsx - News widget
• components/email.tsx - Email widget
• components/parole.tsx - Messages widget
• components/flow.tsx - Tasks widget

API Routes

• app/api/auth/options.ts - NextAuth configuration
• app/api/notifications/ - Notification endpoints
• app/api/courrier/ - Email endpoints
• app/api/calendars/ - Calendar endpoints

---

Document generated: 2024 Last updated: Analysis session