Haven't checked yet
This commit is contained in:
gamer147
405
Documentation/ARCHITECTURE.md
Normal file
405
Documentation/ARCHITECTURE.md
Normal file
@@ -0,0 +1,405 @@
|
||||
# FictionArchive Architecture Overview
|
||||
|
||||
## High-Level Architecture
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────────┐
|
||||
│ React 19 Frontend │
|
||||
│ (Apollo Client, TailwindCSS, OIDC Auth) │
|
||||
└───────────────────────────┬────────────────────────────────────┘
|
||||
│ GraphQL
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────────────┐
|
||||
│ Hot Chocolate Fusion Gateway │
|
||||
│ (FictionArchive.API) │
|
||||
└──────┬────────┬────────┬────────┬────────┬─────────────────────┘
|
||||
│ │ │ │ │
|
||||
▼ ▼ ▼ ▼ ▼
|
||||
┌──────────┐┌──────────┐┌───────────┐┌──────────┐┌──────────────┐
|
||||
│ Novel ││ User ││Translation││Scheduler ││ File │
|
||||
│ Service ││ Service ││ Service ││ Service ││ Service │
|
||||
└────┬─────┘└────┬─────┘└─────┬─────┘└────┬─────┘└──────┬───────┘
|
||||
│ │ │ │ │
|
||||
└───────────┴────────────┴───────────┴─────────────┘
|
||||
│
|
||||
┌────────┴────────┐
|
||||
│ RabbitMQ │
|
||||
│ (Event Bus) │
|
||||
└─────────────────┘
|
||||
│
|
||||
┌────────┴────────┐
|
||||
│ PostgreSQL │
|
||||
│ (per service) │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
## Technology Stack
|
||||
|
||||
| Layer | Technology | Version |
|
||||
|-------|------------|---------|
|
||||
| Runtime | .NET | 8.0 |
|
||||
| GraphQL | Hot Chocolate / Fusion | 13+ |
|
||||
| Database | PostgreSQL | 12+ |
|
||||
| ORM | Entity Framework Core | 8.0 |
|
||||
| Message Broker | RabbitMQ | 3.12+ |
|
||||
| Job Scheduler | Quartz.NET | Latest |
|
||||
| Object Storage | AWS S3 / Garage | - |
|
||||
| Date/Time | NodaTime | Latest |
|
||||
| Frontend | React | 19.2 |
|
||||
| Frontend Build | Vite | 7.2 |
|
||||
| GraphQL Client | Apollo Client | 4.0 |
|
||||
| Auth | OIDC Client TS | 3.4 |
|
||||
| Styling | TailwindCSS | 3.4 |
|
||||
| UI Components | Radix UI | Latest |
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
FictionArchive.sln
|
||||
├── FictionArchive.Common # Shared enums and extensions
|
||||
├── FictionArchive.API # GraphQL Fusion Gateway
|
||||
├── FictionArchive.Service.Shared # Shared infrastructure
|
||||
├── FictionArchive.Service.NovelService
|
||||
├── FictionArchive.Service.UserService
|
||||
├── FictionArchive.Service.TranslationService
|
||||
├── FictionArchive.Service.FileService
|
||||
├── FictionArchive.Service.SchedulerService
|
||||
├── FictionArchive.Service.AuthenticationService
|
||||
├── FictionArchive.Service.NovelService.Tests
|
||||
└── fictionarchive-web # React frontend
|
||||
```
|
||||
|
||||
## Services
|
||||
|
||||
### FictionArchive.API - GraphQL Fusion Gateway
|
||||
|
||||
- **Role**: Single entry point for all GraphQL queries
|
||||
- **Port**: 5001 (HTTPS)
|
||||
- **Endpoints**:
|
||||
- `/graphql` - GraphQL endpoint
|
||||
- `/healthz` - Health check
|
||||
- **Responsibilities**:
|
||||
- Compose GraphQL schemas from all subgraphs
|
||||
- Route queries to appropriate services
|
||||
- CORS policy management
|
||||
|
||||
### FictionArchive.Service.NovelService
|
||||
|
||||
- **Role**: Novel/fiction content management
|
||||
- **Port**: 8081 (HTTPS)
|
||||
- **Database**: `FictionArchive_NovelService`
|
||||
- **GraphQL Operations**:
|
||||
- `GetNovels` - Paginated, filterable novel listing
|
||||
- `ImportNovel` - Trigger novel import
|
||||
- `FetchChapterContents` - Fetch chapter content
|
||||
- **Models**: Novel, Chapter, Source, NovelTag, Image, LocalizationKey
|
||||
- **External Integration**: Novelpia adapter
|
||||
- **Events Published**: `TranslationRequestCreatedEvent`, `FileUploadRequestCreatedEvent`
|
||||
- **Events Subscribed**: `TranslationRequestCompletedEvent`, `NovelUpdateRequestedEvent`, `ChapterPullRequestedEvent`, `FileUploadRequestStatusUpdateEvent`
|
||||
|
||||
### FictionArchive.Service.UserService
|
||||
|
||||
- **Role**: User identity and profile management
|
||||
- **Port**: 8081 (HTTPS)
|
||||
- **Database**: `FictionArchive_UserService`
|
||||
- **Models**: User (with OAuth provider linking)
|
||||
- **Events Subscribed**: `AuthUserAddedEvent`
|
||||
|
||||
### FictionArchive.Service.TranslationService
|
||||
|
||||
- **Role**: Text translation orchestration
|
||||
- **Port**: 8081 (HTTPS)
|
||||
- **Database**: `FictionArchive_TranslationService`
|
||||
- **External Integration**: DeepL API
|
||||
- **Models**: TranslationRequest
|
||||
- **Events Published**: `TranslationRequestCompletedEvent`
|
||||
- **Events Subscribed**: `TranslationRequestCreatedEvent`
|
||||
|
||||
### FictionArchive.Service.FileService
|
||||
|
||||
- **Role**: File storage and S3 proxy
|
||||
- **Port**: 8080 (HTTP)
|
||||
- **Protocol**: REST only (not GraphQL)
|
||||
- **Endpoints**: `GET /api/{*path}` - S3 file proxy
|
||||
- **External Integration**: S3-compatible storage (AWS S3 / Garage)
|
||||
- **Events Published**: `FileUploadRequestStatusUpdateEvent`
|
||||
- **Events Subscribed**: `FileUploadRequestCreatedEvent`
|
||||
|
||||
### FictionArchive.Service.SchedulerService
|
||||
|
||||
- **Role**: Job scheduling and automation
|
||||
- **Port**: 8081 (HTTPS)
|
||||
- **Database**: `FictionArchive_SchedulerService`
|
||||
- **Scheduler**: Quartz.NET with persistent job store
|
||||
- **GraphQL Operations**: `ScheduleEventJob`, `GetScheduledJobs`
|
||||
- **Models**: SchedulerJob, EventJobTemplate
|
||||
|
||||
### FictionArchive.Service.AuthenticationService
|
||||
|
||||
- **Role**: OAuth/OIDC webhook receiver
|
||||
- **Port**: 8080 (HTTP)
|
||||
- **Protocol**: REST only
|
||||
- **Endpoints**: `POST /api/AuthenticationWebhook/UserRegistered`
|
||||
- **Events Published**: `AuthUserAddedEvent`
|
||||
- **No Database** - Stateless webhook handler
|
||||
|
||||
## Communication Patterns
|
||||
|
||||
### GraphQL Federation
|
||||
|
||||
- Hot Chocolate Fusion Gateway composes subgraph schemas
|
||||
- Schema export automated via `build_gateway.py`
|
||||
- Each service defines its own Query/Mutation types
|
||||
|
||||
### Event-Driven Architecture (RabbitMQ)
|
||||
|
||||
- Direct exchange: `fiction-archive-event-bus`
|
||||
- Per-service queues based on `ClientIdentifier`
|
||||
- Routing key = event class name
|
||||
- Headers: `X-Created-At`, `X-Event-Id`
|
||||
- NodaTime JSON serialization
|
||||
|
||||
### Event Flow Examples
|
||||
|
||||
**Novel Import:**
|
||||
```
|
||||
1. Frontend → importNovel mutation
|
||||
2. NovelService publishes NovelUpdateRequestedEvent
|
||||
3. NovelUpdateRequestedEventHandler processes
|
||||
4. Fetches metadata via NovelpiaAdapter
|
||||
5. Publishes FileUploadRequestCreatedEvent (for cover)
|
||||
6. FileService uploads to S3
|
||||
7. FileService publishes FileUploadRequestStatusUpdateEvent
|
||||
8. NovelService updates image path
|
||||
```
|
||||
|
||||
**Translation:**
|
||||
```
|
||||
1. NovelService publishes TranslationRequestCreatedEvent
|
||||
2. TranslationService translates via DeepL
|
||||
3. TranslationService publishes TranslationRequestCompletedEvent
|
||||
4. NovelService updates chapter translation
|
||||
```
|
||||
|
||||
## Data Storage
|
||||
|
||||
### Database Pattern
|
||||
- Database per service (PostgreSQL)
|
||||
- Connection string format: `Host=localhost;Database=FictionArchive_{ServiceName};...`
|
||||
- Auto-migration on startup via `dbContext.UpdateDatabase()`
|
||||
|
||||
### Audit Trail
|
||||
- `AuditInterceptor` auto-sets `CreatedTime` and `LastUpdatedTime`
|
||||
- `IAuditable` interface with NodaTime `Instant` fields
|
||||
- `BaseEntity<TKey>` abstract base class
|
||||
|
||||
### Object Storage
|
||||
- S3-compatible (AWS S3 or Garage)
|
||||
- Path-style URLs for Garage compatibility
|
||||
- Proxied through FileService
|
||||
|
||||
## Frontend Architecture
|
||||
|
||||
### Structure
|
||||
```
|
||||
fictionarchive-web/
|
||||
├── src/
|
||||
│ ├── auth/ # OIDC authentication
|
||||
│ ├── components/ # React components
|
||||
│ │ └── ui/ # Radix-based primitives
|
||||
│ ├── pages/ # Route pages
|
||||
│ ├── layouts/ # Layout components
|
||||
│ ├── graphql/ # GraphQL queries
|
||||
│ ├── __generated__/ # Codegen output
|
||||
│ └── lib/ # Utilities
|
||||
└── codegen.ts # GraphQL Codegen config
|
||||
```
|
||||
|
||||
### Authentication
|
||||
- OIDC via `oidc-client-ts`
|
||||
- Environment variables for configuration
|
||||
- `useAuth` hook for state access
|
||||
|
||||
### State Management
|
||||
- Apollo Client for GraphQL state
|
||||
- React Context for auth state
|
||||
|
||||
## Infrastructure
|
||||
|
||||
### Docker
|
||||
- Multi-stage builds
|
||||
- Base: `mcr.microsoft.com/dotnet/aspnet:8.0`
|
||||
- Non-root user for security
|
||||
- Ports: 8080 (HTTP) or 8081 (HTTPS)
|
||||
|
||||
### Health Checks
|
||||
- All services expose `/healthz`
|
||||
|
||||
### Configuration
|
||||
- `appsettings.json` - Default settings
|
||||
- `appsettings.Development.json` - Dev overrides
|
||||
- `appsettings.Local.json` - Local secrets (not committed)
|
||||
|
||||
---
|
||||
|
||||
# Improvement Recommendations
|
||||
|
||||
## Critical
|
||||
|
||||
### 1. Event Bus - No Dead Letter Queue or Retry Logic
|
||||
**Location**: `FictionArchive.Service.Shared/Services/EventBus/Implementations/RabbitMQEventBus.cs:126-133`
|
||||
|
||||
**Issue**: Events are always ACK'd even on failure. No DLQ configuration for poison messages. Failed events are lost forever.
|
||||
|
||||
**Recommendation**: Implement retry with exponential backoff, dead-letter exchange, and poison message handling.
|
||||
|
||||
```csharp
|
||||
// Example: Add retry and DLQ
|
||||
catch (Exception e)
|
||||
{
|
||||
_logger.LogError(e, "Error handling event");
|
||||
if (retryCount < maxRetries)
|
||||
{
|
||||
await channel.BasicNackAsync(@event.DeliveryTag, false, true); // requeue
|
||||
}
|
||||
else
|
||||
{
|
||||
// Send to DLQ
|
||||
await channel.BasicNackAsync(@event.DeliveryTag, false, false);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. CORS Configuration is Insecure
|
||||
**Location**: `FictionArchive.API/Program.cs:24-33`
|
||||
|
||||
**Issue**: `AllowAnyOrigin()` allows requests from any domain, unsuitable for production.
|
||||
|
||||
**Recommendation**: Configure specific allowed origins via appsettings:
|
||||
```csharp
|
||||
builder.Services.AddCors(options =>
|
||||
{
|
||||
options.AddPolicy("Production", policy =>
|
||||
{
|
||||
policy.WithOrigins(builder.Configuration.GetSection("Cors:AllowedOrigins").Get<string[]>())
|
||||
.AllowAnyMethod()
|
||||
.AllowAnyHeader();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### 3. Auto-Migration on Startup
|
||||
**Location**: `FictionArchive.Service.Shared/Services/Database/FictionArchiveDbContext.cs:23-38`
|
||||
|
||||
**Issue**: Running migrations at startup can cause race conditions with multiple instances and potential data corruption during rolling deployments.
|
||||
|
||||
**Recommendation**: Use a migration job, init container, or CLI tool instead of startup code.
|
||||
|
||||
## Important
|
||||
|
||||
### 4. No Circuit Breaker Pattern
|
||||
**Issue**: External service calls (DeepL, Novelpia, S3) lack resilience patterns.
|
||||
|
||||
**Recommendation**: Add Polly for circuit breaker, retry, and timeout policies:
|
||||
```csharp
|
||||
builder.Services.AddHttpClient<ISourceAdapter, NovelpiaAdapter>()
|
||||
.AddPolicyHandler(GetRetryPolicy())
|
||||
.AddPolicyHandler(GetCircuitBreakerPolicy());
|
||||
```
|
||||
|
||||
### 5. Missing Request Validation/Rate Limiting
|
||||
**Issue**: No visible rate limiting on GraphQL mutations. `ImportNovel` could be abused.
|
||||
|
||||
**Recommendation**: Add rate limiting middleware and input validation.
|
||||
|
||||
### 6. Hardcoded Exchange Name
|
||||
**Location**: `RabbitMQEventBus.cs:24`
|
||||
|
||||
**Issue**: `fiction-archive-event-bus` is hardcoded.
|
||||
|
||||
**Recommendation**: Move to configuration for environment flexibility.
|
||||
|
||||
### 7. No Distributed Tracing
|
||||
**Issue**: Event correlation exists (`X-Event-Id` header) but not integrated with tracing.
|
||||
|
||||
**Recommendation**: Add OpenTelemetry for end-to-end request tracing across services.
|
||||
|
||||
### 8. Singleton AuditInterceptor
|
||||
**Location**: `FictionArchiveDbContext.cs:20`
|
||||
|
||||
**Issue**: `new AuditInterceptor()` created per DbContext instance.
|
||||
|
||||
**Recommendation**: Register as singleton in DI and inject.
|
||||
|
||||
## Minor / Code Quality
|
||||
|
||||
### 9. Limited Test Coverage
|
||||
**Issue**: Only `NovelService.Tests` exists. No integration tests for event handlers.
|
||||
|
||||
**Recommendation**: Add unit and integration tests for each service, especially event handlers.
|
||||
|
||||
### 10. Inconsistent Port Configuration
|
||||
**Issue**: Some services use 8080 (HTTP), others 8081 (HTTPS).
|
||||
|
||||
**Recommendation**: Standardize on HTTPS with proper cert management.
|
||||
|
||||
### 11. No API Versioning
|
||||
**Issue**: GraphQL schemas have no versioning strategy.
|
||||
|
||||
**Recommendation**: Consider schema versioning or deprecation annotations for breaking changes.
|
||||
|
||||
### 12. Frontend - No Error Boundary
|
||||
**Issue**: React app lacks error boundaries for graceful failure handling.
|
||||
|
||||
**Recommendation**: Add React Error Boundaries around routes.
|
||||
|
||||
### 13. Missing Health Check Depth
|
||||
**Issue**: Health checks only verify service is running, not dependencies.
|
||||
|
||||
**Recommendation**: Add database, RabbitMQ, and S3 health checks:
|
||||
```csharp
|
||||
builder.Services.AddHealthChecks()
|
||||
.AddNpgSql(connectionString)
|
||||
.AddRabbitMQ()
|
||||
.AddS3(options => { });
|
||||
```
|
||||
|
||||
### 14. Synchronous File Operations in Event Handlers
|
||||
**Issue**: File uploads may block event handling thread for large files.
|
||||
|
||||
**Recommendation**: Consider async streaming for large files.
|
||||
|
||||
## Architectural Suggestions
|
||||
|
||||
### 15. Consider Outbox Pattern
|
||||
**Issue**: Publishing events and saving to DB aren't transactional, could lead to inconsistent state.
|
||||
|
||||
**Recommendation**: Implement transactional outbox pattern for guaranteed delivery:
|
||||
```
|
||||
1. Save entity + outbox message in same transaction
|
||||
2. Background worker publishes from outbox
|
||||
3. Delete outbox message after successful publish
|
||||
```
|
||||
|
||||
### 16. Gateway Schema Build Process
|
||||
**Issue**: Python script (`build_gateway.py`) for schema composition requires manual execution.
|
||||
|
||||
**Recommendation**: Integrate into CI/CD pipeline or consider runtime schema polling.
|
||||
|
||||
### 17. Secret Management
|
||||
**Issue**: Credentials in appsettings files.
|
||||
|
||||
**Recommendation**: Use Azure Key Vault, AWS Secrets Manager, HashiCorp Vault, or similar secret management solution.
|
||||
|
||||
---
|
||||
|
||||
## Key Files Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `FictionArchive.API/Program.cs` | Gateway setup |
|
||||
| `FictionArchive.API/build_gateway.py` | Schema composition script |
|
||||
| `FictionArchive.Service.Shared/Services/EventBus/` | Event bus implementation |
|
||||
| `FictionArchive.Service.Shared/Extensions/` | Service registration helpers |
|
||||
| `FictionArchive.Service.Shared/Services/Database/` | DB infrastructure |
|
||||
| `fictionarchive-web/src/auth/AuthContext.tsx` | Frontend auth state |
|
||||
Reference in New Issue
Block a user