Saturday, February 28, 2026

Onion / Clean Architecture Deep Dive

 Both Clean Architecture and Onion Architecture follow the same core rule:

🔑 Dependencies always point inward toward the Domain.



Onion / Clean Architecture – Layer Overview

Typical layers (from inside to outside):

[ Domain ] ← pure business logic
[ Application ] ← use cases
[ Infrastructure ] ← database, external services
[ Presentation ] ← controllers, API, UI

Example: Employee Management System

We’ll implement:

“Create a new Employee and save it.”


Domain Layer (Core Business Logic)

📁 Domain/

This is the center of the onion.

Where to define domain objects?

👉 HERE.

Domain objects like:

  • Employee

  • Order

  • Product

  • Customer

must live in Domain.

They must NOT depend on:

  • Database

  • Frameworks

  • Controllers

  • UI


Domain Entity

📁 Domain/Entities/Employee.cs

namespace Domain.Entities
{
public class Employee
{
public Guid Id { get; private set; }
public string Name { get; private set; }
public decimal Salary { get; private set; }

public Employee(string name, decimal salary)
{
if (salary <= 0)
throw new ArgumentException("Salary must be greater than zero");

Id = Guid.NewGuid();
Name = name;
Salary = salary;
}

public void IncreaseSalary(decimal amount)
{
if (amount <= 0)
throw new ArgumentException("Increase must be positive");

Salary += amount;
}
}
}

✔ Business rules live here
✔ No EF, no DB, no HTTP


Repository Interface (Contract)

📁 Domain/Interfaces/IEmployeeRepository.cs

namespace Domain.Interfaces
{
public interface IEmployeeRepository
{
void Add(Employee employee);
Employee GetById(Guid id);
}
}

IMPORTANT:

The interface lives in Domain, not Infrastructure.

Why?

Because Domain defines what it needs.
Infrastructure will implement it.

This keeps dependencies inward.


Application Layer (Use Cases)

📁 Application/

This layer orchestrates business operations.

Where to implement use cases?

👉 HERE.

Example use case:

  • CreateEmployee

  • IncreaseSalary

  • GetEmployee


Create Employee Use Case

📁 Application/UseCases/CreateEmployee.cs

using Domain.Entities;
using Domain.Interfaces;

namespace Application.UseCases
{
public class CreateEmployee
{
private readonly IEmployeeRepository _repository;

public CreateEmployee(IEmployeeRepository repository)
{
_repository = repository;
}

public Guid Execute(string name, decimal salary)
{
var employee = new Employee(name, salary);
_repository.Add(employee);

return employee.Id;
}
}
}

✔ Uses Domain entity
✔ Uses Domain interface
✔ Does NOT know about database


Infrastructure Layer (External World)

📁 Infrastructure/

This is where:

  • EF Core

  • SQL

  • MongoDB

  • File system

  • External APIs

live.


Where to implement repository?

👉 HERE.


EF Repository Implementation

📁 Infrastructure/Repositories/EmployeeRepository.cs

using Domain.Entities;
using Domain.Interfaces;

namespace Infrastructure.Repositories
{
public class EmployeeRepository : IEmployeeRepository
{
private readonly AppDbContext _context;

public EmployeeRepository(AppDbContext context)
{
_context = context;
}

public void Add(Employee employee)
{
_context.Employees.Add(employee);
_context.SaveChanges();
}

public Employee GetById(Guid id)
{
return _context.Employees.Find(id);
}
}
}

✔ Implements Domain interface
✔ Depends on EF
✔ Domain does NOT depend on this

Dependency direction:

Infrastructure → Domain

Correct ✅


Presentation Layer (API / UI)

📁 WebAPI/Controllers/EmployeeController.cs


Where to call the use case?

👉 HERE (Presentation Layer)


using Application.UseCases;
using Microsoft.AspNetCore.Mvc;

[ApiController]
[Route("api/employees")]
public class EmployeeController : ControllerBase
{
private readonly CreateEmployee _createEmployee;

public EmployeeController(CreateEmployee createEmployee)
{
_createEmployee = createEmployee;
}

[HttpPost]
public IActionResult Create(string name, decimal salary)
{
var id = _createEmployee.Execute(name, salary);
return Ok(id);
}
}

✔ Controller calls Use Case
✔ Controller does NOT create Employee directly
✔ Controller does NOT talk to DB


Summary – Clear Answers
❓ Where to define domain objects like Employee or Order?

Domain Layer
Domain/Entities/Employee.cs
Domain/Entities/Order.cs

They contain:

  • Business rules
  • Validation
  • Core behavior

They must NOT know:

  • Database
  • UI
  • Frameworks


❓ Where to implement repository?

  • Interface → Domain
  • Implementation → Infrastructure

Domain/Interfaces/IEmployeeRepository
Infrastructure/Repositories/EmployeeRepository

❓ Where to implement business logic?

  • Core rules → Domain
  • Use case orchestration → Application


❓ Where to call use cases?

From:

  • Controllers
  • API endpoints
  • UI
  • Background jobs

That’s the Presentation Layer.


Dependency Rule (Most Important)

Only this direction is allowed:

Presentation → Application → Domain
Infrastructure → Domain

🚫 Domain must never depend on outer layers.


Clean vs Onion?

Practically identical in structure.

  • Onion Architecture was popularized by Jeffrey Palermo
  • Clean Architecture was formalized by Robert C. Martin

Both enforce:

“Business rules should not depend on frameworks.”

Wednesday, February 25, 2026

AWS | Trusted Advisor

An advisor service that scans your resources and infrastructure compare them with best practices and provide recommendations on important aspects like cost, performance, security and fault tolerance.





Tuesday, February 24, 2026

AWS | Athena Vs S3 Select Vs Redshift Spectrum

Comparison between Athena, S3 Select and Redshift Spectrum

FeatureAmazon S3 SelectAmazon AthenaAmazon Redshift Spectrum
What it isQuery individual objects in S3Serverless SQL query engine on S3Query S3 from Redshift
Best forApp-level filtering of single filesAd-hoc analyticsEnterprise data warehouse extension
SetupNone (API call)None (serverless)Requires Redshift cluster
SQL supportLimited (simple SQL)Full ANSI SQLFull Redshift SQL
PerformanceGood for small object filteringGood for medium-large datasetsBest for very large datasets
Pricing modelPer data scannedPer TB scannedPer TB scanned + Redshift cost
ConcurrencyApp-controlledHighVery high
Use case exampleFetch specific rows from JSON/CSV in appRun analytics on data lakeJoin S3 data with warehouse tables

When to Use What

Use S3 Select when:
  • You need to retrieve specific rows from one object
  • You're inside an application
  • You want to reduce data transfer

Think: “Filter before downloading.”


Use Athena when:
  • You have a data lake in S3
  • You want SQL without managing infrastructure
  • You need BI / analytics

Think: “Serverless analytics on S3.”


Use Redshift Spectrum when:
  • You already use Redshift
  • You want to join warehouse tables + S3 data
  • You need enterprise-scale performance

Think: “Extend data warehouse to S3.”


Simple Decision Rule
  • Single file → S3 Select
  • Data lake analytics → Athena
  • Enterprise warehouse + S3 → Redshift Spectrum

Athena Vs S3 Select

Amazon Athena

A serverless interactive query service that lets you run full SQL queries directly on data stored in S3.

Amazon S3 Select

A feature of S3 that lets you retrieve only a subset of data from a single object using simple SQL expressions.


Key Differences

FeatureAthenaS3 Select
ScopeQuery across multiple filesQuery within a single object
SQL SupportFull ANSI SQLLimited SQL (simple SELECT, WHERE)
Use CaseAnalytics, reporting, BIEfficient object-level filtering
PerformanceScans full dataset (optimized by partitioning)Reads only selected data from object
PricingPer TB scannedPer GB scanned + data returned
SchemaRequires table definition (Glue/Data Catalog)No external catalog needed
JoinsYesNo
AggregationsYesVery limited

When to Use Each

Use Athena when:
  • You need to query large datasets across many files
  • You need joins, aggregations, grouping
  • You're connecting BI tools (e.g., QuickSight)
  • You want serverless analytics without managing infrastructure

Example:

SELECT customer_id, SUM(amount)
FROM transactions
GROUP BY customer_id;

Use S3 Select when:

  • You need to fetch a small portion of a single large file
  • You want to reduce network transfer
  • You’re building an application that reads filtered object data
  • You need low-latency object-level filtering

Example:

SELECT * FROM s3object s WHERE s.status = 'active'

Cost Consideration

  • Athena can become expensive if queries scan large unpartitioned datasets.
  • S3 Select is often cheaper when extracting small pieces of large objects.


Simple Mental Model

  • Athena = Data warehouse-style querying over S3
  • S3 Select = Smart “grep” inside one S3 file

Saturday, February 21, 2026

Web API - Caching

Note: Response/Output cache handled by their respective middlewares and do not hit controller in subsequent requests until cache invalidated.
Data Cache (IMemorycache, Memcache, Redis): Managed by code, you check in code if data available in cache then serve else reproduce and serve.

1. Output Caching (Recommended for .NET 7+)

Starting in ASP.NET Core 7+, Microsoft introduced built-in Output Caching Middleware, which is the best way to cache full API responses.

Step 1: Register Output Caching

In Program.cs:

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOutputCache();

var app = builder.Build();

app.UseOutputCache();

app.MapControllers();

app.Run();

Step 2: Apply Caching to an Endpoint
[HttpGet]
[OutputCache(Duration = 60)]
public IActionResult GetProducts()
{
return Ok(GetProductsFromDatabase());
}

🔹 This caches the response for 60 seconds.
🔹 Works great for GET endpoints.


Advanced Example (Custom Policy)
builder.Services.AddOutputCache(options =>
{
options.AddPolicy("MyPolicy", policy =>
policy.Expire(TimeSpan.FromMinutes(5))
.SetVaryByQuery("category"));
});

Then:

[OutputCache(PolicyName = "MyPolicy")]

2. Response Caching (HTTP Cache Headers)

This uses HTTP headers like Cache-Control. It works with browser/proxy caching.

Enable Middleware
builder.Services.AddResponseCaching();
app.UseResponseCaching();
Use Attribute
[ResponseCache(Duration = 60)]
[HttpGet]
public IActionResult GetProducts()
{
return Ok(data);
}

🔹 This sets HTTP headers.
🔹 It does NOT cache on the server by default.
🔹 Good for public APIs.


3. Data Cache 
In-Memory Caching (IMemoryCache)

Best when you want to cache data, not full HTTP responses.

Register
builder.Services.AddMemoryCache();
Use in Controller
private readonly IMemoryCache _cache;

public ProductsController(IMemoryCache cache)
{
_cache = cache;
}

[HttpGet]
public IActionResult GetProducts()
{
if (!_cache.TryGetValue("products", out List<Product> products))
{
products = GetProductsFromDatabase();

var cacheOptions = new MemoryCacheEntryOptions()
.SetSlidingExpiration(TimeSpan.FromMinutes(5));

_cache.Set("products", products, cacheOptions);
}

return Ok(products);
}

4. Distributed Caching (Redis, SQL Server)

For multi-server environments, use distributed cache.

Common choice:

  • Redis: Redis can store complex data types like Lists, arrays, Hash tables etc.

  • Memcached: Simple text string store 

Example Setup with Redis
builder.Services.AddStackExchangeRedisCache(options =>
{
options.Configuration = "localhost:6379";
});

Then inject IDistributedCache.


FeatureMemcachedRedis
DistributedYesYes
ReplicationNo (basic)Yes
PersistenceNoYes
Data loss riskHigherLower
ComplexitySimpleAdvanced



Which One Should You Use?

ScenarioRecommended
Cache full API response    OutputCache
Scale across multiple servers    Redis (Distributed Cache)
Cache DB results only    IMemoryCache
Browser/proxy caching    ResponseCache

Best Practice Tips
  • Cache only GET requests.
  • Avoid caching user-specific data unless using VaryBy.
  • Always define expiration.
  • Invalidate cache on data update.
  • Use distributed cache in production with multiple instances.

Output cache and Response Cache both stores whole response handled by middleware, only difference is storage location. 

Response Caching vs Output Caching
FeatureResponse CachingOutput Caching
Where caching happensClient / Proxy (browser, CDN)Server (inside ASP.NET Core)
MiddlewareUseResponseCaching()UseOutputCache()
Actually stores response on server?NoYes
Performance benefit for serverMinimalSignificant
Best forPublic APIsHigh-performance APIs
IntroducedEarly ASP.NET Core versions.NET 7

Friday, February 13, 2026

CI/CD - Safe DB Changes/Migrations

Safe DB Migrations means updating your database schema without breaking the running application and without downtime.

In real systems (AWS, Docker, microservices, CI/CD), DB changes are riskier than code changes because data is permanent and shared by multiple services.

Why This Matters in CI/CD & Microservices
  • Multiple services share DB
  • Rolling deployments mean old + new versions run together
  • Containers auto-scale
  • Instant rollback must be possible

Unsafe DB change = system outage.

“Safe DB migrations involve backward-compatible schema updates, using the expand-migrate-contract pattern, enabling changes through feature flags, and executing migrations before application rollout to avoid downtime and ensure old and new versions can run simultaneously.”

DB Migration(Keep old schema + new schema both, this will keep old and newly deployed app both working)→ Deploy new App -> Test new app → Enable Feature flag to take new app in effect to users-> delete old schema



1. Backward-Compatible Schema Changes

Goal: Old app version + New app version should both work during deployment.

Bad Example 
DROP COLUMN FirstName;

Old app still expects FirstName → crash.

Good Example 

ALTER TABLE Users ADD COLUMN FullName;

Old app still works, new app starts using new column.

Rule:
Never delete or rename columns immediately.


2. Feature Flags

A feature flag lets you deploy DB + code but activate behavior later.

Example flow:

  • Add new column FullName
  • Deploy code that can use it
  • Keep feature OFF
  • Turn feature ON after validation

This reduces risk and allows instant rollback without DB rollback.


3. Expand → Migrate → Contract Pattern

This is the gold standard for safe DB migrations.

Phase 1 — Expand

Add new schema without removing old.

ALTER TABLE Users ADD COLUMN FullName;

Both old and new apps still work.


Phase 2 — Migrate

Move data gradually.

UPDATE Users SET FullName = FirstName + ' ' + LastName;

Background job, no downtime.


Phase 3 — Contract

After all services use new column:

DROP COLUMN FirstName;
DROP COLUMN LastName;

Now safe because nothing depends on them.


4. Run Migrations Before App Rollout

Correct order:

DB Migration → Deploy App → Enable Feature

Wrong order:

Deploy App → Migration Later → Crash Risk

If app expects a column that doesn’t exist yet → production failure.


Ideal Safe Migration Checklist
PracticeWhy
Backward compatible schemaPrevent crashes
Expand-Migrate-ContractZero downtime
Feature flagsSafe activation
Pre-deployment migrationsAvoid missing schema
Data backfill jobsNon-blocking
Rollback planDisaster recovery

Thursday, February 12, 2026

CI/CD - Ideal Pipeline Characteristics

 An Ideal CI/CD Pipeline with Harness means an automated flow where code goes from developer → build → test → security → deploy → monitor with minimum manual work and strong governance.

Think of it as a production assembly line for software.


High-Level Flow
Developer → Git Push → CI Build → Test → Security → Artifact → CD Deploy → Monitor → Feedback

Harness mainly shines in Continuous Deployment, but it also supports CI.

“An ideal CI/CD pipeline with Harness automates build, test, security scanning, artifact versioning, progressive deployment strategies like canary or blue-green, governance approvals, and automated rollback with continuous monitoring across environments.”




Stage-by-Stage Ideal Pipeline

1. Source Control (Git)

Tools: GitHub / GitLab / Bitbucket

What happens

  • Developer pushes code
  • Pull request triggers pipeline

Best Practice

  • Branch strategy (main / develop / feature)
  • Mandatory code review


2. Continuous Integration (CI)

Steps

  1. Checkout Code
  2. Build
  3. Unit Tests
  4. Code Quality
  5. Security Scan
  6. Create Artifact (Docker / JAR / DLL)

Typical Tools
  • Build: .NET CLI / Maven / npm
  • Quality: SonarQube
  • Security: Snyk / Trivy
  • Artifact Repo: Nexus / Artifactory / ECR

Goal: Ensure every commit is buildable and safe.


3. Artifact Storage

Store versioned outputs:

  • Docker Images → AWS ECR
  • NuGet / JARs → Artifact repo

Key Rule:
Only deploy immutable artifacts, never rebuild in CD.


Continuous Deployment (Harness Strength Area)

4. Environment Promotion Flow
Dev → QA → Staging → Prod

Harness provides:

  • Visual pipeline
  • Rollback automation
  • Approval gates
  • Feature flags
  • Canary / Blue-Green deploy


5. Deployment Strategies

Blue-Green

  • Old version live
  • New version parallel
  • Switch traffic instantly

Canary

  • 5% → 25% → 50% → 100%
  • Auto rollback on failure

Rolling
  • Replace containers gradually

Harness automates health checks and rollback without manual SSH.


6. Approvals & Governance

Before Prod:

  • Manual approval
  • Security approval
  • Change-management integration
  • RBAC controls

This is crucial for enterprises.


7. Infrastructure Integration

Works with:

  • Kubernetes (EKS)
  • ECS / Fargate
  • VMs
  • Serverless (Lambda)
  • Terraform / CloudFormation

Harness doesn’t replace infra tools — it orchestrates them.


8. Monitoring & Feedback

After deployment:

  • Metrics (CPU, memory, error rate)
  • Logs
  • APM tools
  • Automated rollback triggers

This closes the DevOps loop.


Ideal Pipeline Characteristics

CharacteristicWhy Important
AutomatedRemoves human error
FastShort feedback cycle
SecureScans before deploy
ImmutablePredictable releases
ObservableQuick failure detection
Rollback ReadyReduces risk
Multi-EnvDev → Prod consistency

Example Ideal Flow (AWS + .NET + Docker)

  1. Developer pushes code
  2. CI builds Docker image
  3. Unit + security tests
  4. Push to ECR
  5. Harness picks image
  6. Deploy to Dev ECS
  7. Auto test
  8. Promote to QA
  9. Manual approval
  10. Canary to Prod
  11. Auto health check
  12. Rollback if error > threshold

Node | Cluster Vs Worker Threads

Cluster: Multiple processes (scale app across CPU cores) Worker Threads: Multiple threads (handle CPU-heavy work inside one process) Cluster...