AZ - Cosmos DB

Today’s applications are required to be highly responsive and always online. To achieve low latency and high availability, instances of these applications need to be deployed in data centers that are close to their users. Applications need to respond in real-time to large changes and make this data available to users in milliseconds.

Azure Cosmos DB is Microsoft's globally distributed, multi-model database service. With a click of a button, Cosmos DB enables you to elastically and independently scale throughput and storage across any number of Azure regions worldwide. You can elastically scale throughput and storage, and take advantage of fast, single-digit-millisecond data access using your favorite API.
Cosmos DB provides comprehensive service level agreements (SLAs) for throughput, latency, availability, and consistency guarantees, something no other database service offers.

Databases Supported in CosmosDB

1. SQL
2. MongoDB
3. Cassandra
4. Azure Tables
5. Gremlin (Graph) 

Key Benefits 

1. Global Distribution of Data: Cosmos DB enables you to build highly responsive and highly available applications worldwide. Cosmos DB transparently replicates your data wherever your users are, so your users can interact with a replica of the data that is closest to them.
   Cosmos DB allows you to add or remove any of the Azure regions to your Cosmos account at any time, with a click of a button. Cosmos DB will seamlessly replicate your data to all the regions associated with your Cosmos account while your application continues to be highly available.
2. Highly Available: 99.999%
3. Scalable Throughput and Storage
4. Low Latency
   It guarantees less than 10 ms response time throughout the world
5. Five consistency choices
   



• Strong: This means that reads are guaranteed to see the most recent write
  Clients get the old value from all read regions until and unless a new committed value to write region synced to all read regions.
  After the point of time when it confirmed from all regions that value gets synced clients start to receive the new value.
  Ensures order of data client receives 
  
• Bounded staleness: Most frequently chosen by globally distributed applications expecting low write latencies but total global order guarantees
  All read regions get synced at a specified time lag, till that time all-regions provide old value, after the specified point of time all regions provide new value.
  Ensures order of data client receives 
• Session: Session consistency is most widely used consistency level both for single region as well as, globally distributed applications
  In the case of the distributed DB across regions, the client gets the value whatever present on the DB of the region it has started the session.
  When a new value updated in write region client of any read region will get that value when that will sync in clients DB region(where the client made his session).
  However, a client who committed new value will start to get new value from the time he committed
  Ensures order of data client receives
• Consistent Prefix: Guarantees that reads never see out of order writes
  High performance like eventual
  Ensures order of data client receives
• Eventual: The weakest form of consistency wherein a client may get the values which are older than the ones it had seen before, over time
  Does not ensure order of data client receives
  
6. Schema & Index management
   Keeping database schema and indexes in-sync with an application’s schema is especially painful for globally distributed apps. With Cosmos DB, you do not need to deal with schema or index management.
7. Battle Tested: Microsoft's mission-critical applications use it
8. Global Presence: 54+ regions globally
9. Secured: Data is encrypted at rest and in motion
10. Fully Managed: You no need to worry about managing deployments to multi-data centers its taken care by Azure with licensing you opt.  
11. Sparks: You can run Spark directly on data stored in Cosmos DB. This capability allows you to do low-latency, operational analytics at global scale without impacting transactional workloads operating directly against Cosmos DB

AZR - Function App

 Azure Functions provides "compute on-demand" - and in two significant ways.

First, Azure Functions allows you to implement your system's logic into readily available blocks of code. These code blocks are called "functions". Different functions can run anytime you need to respond to critical events.

Second, as requests increase, Azure Functions meets the demand with as many resources and function instances as necessary - but only while needed. As requests fall, any extra resources and application instances drop off automatically.

Azure Functions has 3 pricing plans

• Consumption plan: Azure provides all of the necessary computational resources. You don't have to worry about resource management, and only pay for the time that your code runs.
  -Pay only when your function(s) are running
  -Scale out/in as and when required 

• Premium plan: You specify a number of pre-warmed instances that are always online and ready to immediately respond. When your function runs, Azure provides any additional computational resources that are needed. You pay for the pre-warmed instances running continuously and any additional instances you use as Azure scales your app in and out.

• App Service plan: Run your functions just like your web apps. If you use App Service for your other applications, your functions can run on the same plan at no additional cost.

          Session Affinity

  

AZR - Virtual Machine Scale Set

(Azure Scale Set is like Autoscaling group in AWS)

Azure virtual machine scale sets enable you to create and manage a group of identical, load-balanced VMs. The number of VM instances can automatically increase or decrease in response to demand or a defined schedule.
Scale sets provide high availability to your applications and allow you to centrally manage, configure, and update a large number of VMs.
With virtual machine scale sets, you can build large-scale services for areas such as compute, big data, and container workloads.


Why Scale Set

• Easy to create & maintain multiple VMs: Easy scaling of hundreds of VMs because all created by the same OS image.
• Networking: Uses 'Load Balancers' with Basic layer-4 traffic distribution and 'Azure Application Gateway' with advanced layer-7 traffic distribution.
• Load Balancers: Used for IP/port based mapping with VMs
• Azure Application Gateway: Used for URL based mapping with VMs

• Provides high availability and application resiliency: With the help of multiple VMs using Availability set or Availability zones.
• Support Spot Instances: You can set Spot price that is a maximum price per hour US $ you want to pay for an instance. Azure allot instance if your set price is greater than the platform price at that time.

• Auto Scaling: Provide auto scaling

• Large Scale handling: 
• Scale sets support up to 1,000 VM instances. If you create and upload your own custom VM images, the limit is 600 VM instances. 
• For the best performance with production workloads, use Azure Managed Disks.
• Support additional disk for VMs

Features

1. Control it like IaaS, scale it like PaaS: Deploy Virtual Machine Scale Sets using Azure Resource Manager (ARM) templates with support for Windows and Linux platform images, as well as custom images and extensions.
2. Run Cassandra, Cloudera, Hadoop, MongoDB, and Mesos
3. Quickly scale your big compute and big data applications
4. Attach additional data disks as per your application requirement
   

Azure - Queues

Azure Queue Storage provides cloud messaging between application components. In designing applications for scale, application components are often decoupled, so that they can scale independently. Queue storage delivers asynchronous messaging for communication between application components, whether they are running in the cloud, on the desktop, on an on-premises server, or on a mobile device. Queue storage also supports managing asynchronous tasks and building process workflows.

It is a FIFO approach

Storage Account > Queue > Messages

Important Classes & Methods

Class: CloudStorageAccount
Method: CreateCloudBlobClient

Class: CloudQueueClient

Class:CloudQueue
Method: CreateIfNotExist
Method: PeekMessge
Method: UpdateMessge
Method: DeleteMessge

---Dequeue--
Method:GetMessage
Method:GetMessages - Read all visible messages or no. of messages you passed as parameter of the queue.

Message visibility to other clients, time is 30 sec by default. While Fetching message with GetMessage or updating with UpdateMessage you can change default visibility time and set as you wish, with passing a TimeSpan object as a parameter.

A message must be processed by one client only 




Dequeue a message
GetMessage   >  Process Message > Delete Message

GetMessage: Fetch message and block that message to other clients that means it will not visible to other clients for visibility time.
You need to process and delete this message before visible time finishes. Because after that, the message will be visible to other clients and another client may block that.

PeekMessage: It returns first available message of queue and do not block that message like GetMessage method.
It means parallel other clients may also read the same message. 

Class: CloudQueueMessage
Method: SetMessageContent
Property: Id
Property: PopupReceipt

Azure - Azure Storage Table (AZ)

Azure tables are ideal for storing structured, non-relational data. Common uses of Table storage include:

• Storing TBs of structured data capable of serving web scale applications
• Storing datasets that don't require complex joins, foreign keys, or stored procedures and can be denormalized for fast access
• Quickly querying data using a clustered index
• Accessing data using the OData protocol and LINQ queries with WCF Data Service .NET Libraries
  
  Retreive Entity
  TableOperation TO = TableOperation.Retreive(PartitionKey, Rowkey);
  TableResult TR = TableEmp.Execute(TO);
  EmpEntity emp = TR.Result;
  
  Update Entity
  Update proprties of emp
  TableOperation TO = TableOperation.Replace(emp);
  TableEmp.Execute(TO)
  
  Delete Entity
  TableOperation TO = TableOperation.Delete(Entity)
  TableEmp.Execute(TO)
  
  
  Optimization Techniques
  
  

1. Read First: Read first the entity using Partition name + Row key
2. Multiple Keys: Keep multiple keys, if data is duplicating no worries
3. Compound Key: You can make Row key as a compound key
   Ex. If you store 2 values (Id and Email) in Row key, you can search with any of the mob. or email, this is a compound key. Id_<Id> and Email_<Email>
   
   

   PartitionKey	RowKey	EmpName
   Employee	Id_1001	Megha
   Employee	Id_1002	Renuka
   Employee	Email_tomar@gmail.com	Tomar
   Employee	Email_mukesh12@gmail.com	Mukesh

4. Avoid unnecessary tables: Try to keep all related entities in one table separated by Partition key. Makes transactions smooth (commit/rollback)
   Ex. Emp, EmpDetails
5. Inter Partition Pattern: Keeping multiple type values in row key
   Keeping multiple values to divide search load, like people searching with email id will search with a key like "Email_ %"
   
   Compound Key example (point no.3) is an Inter Partition pattern example.
6. Intra Partition Pattern: Dividing search by using multiple Partition key is Intra Partition Pattern.
   
   

   PartitionKey	RowKey	EmpName
   EmployeeId	1001	Megha
   EmployeeId	1002	Renuka
   EmployeeEmail	tomar@gmail.com	Tomar
   EmployeeEmail	mukesh12@gmail.com	Mukesh

    
7. Delete Partition Pattern: This enables bulk delete. When you delete data based on the partition key.
   Ex. you can delete any month data in a single operation.
   
   

   PARTITIONKEY	ROWKEY	EMPNAME
   EMPLOYEE-JAN20	1001	Megha
   EMPLOYEE-JAN20	1002	Renuka
   EMPLOYEE-JAN20	1003	Tomar
   EMPLOYEE-FEB20	1004	Mukesh
   EMPLOYEE-FEB20	1005	Kailash

   
   
8. Large Entity Pattern: In case you are storing image/binary data you can use blog to store 
9. Long table Pattern: In case you have large no. of columns in your entity
   

Azure - Azure Storage (AZ)

Azure categories storage items in 4 categories

1. File    
   Used for files storage like text file, word file, pdf file, etc.
2. Blob 
   Used for the binary data store like an image file or library files etc.
3. Table
   Used to store key-value pairs
4. Queue
   Used to store queue messages. It works in a FIFO manner.

Account Kind of Storage

1. Storage (General Purpose v1)
   A general-purpose and used for legacy deployable (stuff build before 2014) that can be used to store file, blob, table and the queue.
2. StorageV2 (General Purpose v2)
   Recommended as it has the latest features and option to choose Access Tier as well
   A general-purpose and used to store file, blob, table and the queue.
3. Blob Storage
   Storage accounts with premium performance characteristics for block blobs and append blobs. Recommended for scenarios with high transaction rates, or scenarios that use smaller objects or require consistently low storage latency.

Replication or Data Redundancy

There are multiple options available for your requirements of Durability and High Availability

1. LRS(Locally Redundant Storage)
   Stores 3 copies of your data locally in a single physical location synchronously of the primary region.
   Cheapest option
   Not recommended for applications required high availability
2. ZRS(Zone-Redundant Storage)
   Copies your data synchronously across three Azure availability zones in the primary region. For applications requiring high availability. with economic rates.
   Durability: 99.99999999999 (11 9's)
3. GRS(Geo Redundant Storage)
   Copies 3 local copies synchronously using LRS of your data in the primary region and in a different geo-location asynchronously in the secondary region. you can think it as Geo Locally Redundant Storage.
   Durability: 99.999999999999 (12 9's)
4.  GZRS(Geo-Zone-Redundant Storage)
   Copies data in 3 Azure Availability zones synchronously in the primary region and in different Geo location asynchronously in the secondary region.
   Durability: 99.9999999999999999 (16 9's)
5. RA-GRS(Read Access Geo Redundant Access) Not supported currently
   Copies 3 local copies synchronously using LRS of your data in the primary region and in a different geo-location asynchronously in the secondary region. you can think it as Geo Locally Redundant Storage with reading access to secondary region data.
   Secondary region data is available to read in case your primary region unavailable.
6. RA-GZRS(Read Access Geo Zone Redundant Access) Not supported currently
   Copies data in 3 Azure Availability zones synchronously in the primary region and in different Geo location asynchronously in the secondary region. This replication avail read access to secondary region data. You can access secondary region data in case of primary unavailable.
   Durability: 99.9999999999999999 (16 9's)

Performance
This section basically define disk type which would use to store data

• Standard: Data backed into magnetic HDD drives, it offers cheap rates.  
• Premium: Data backed into solid-state drives SDD, provides high IOPS rate with 99.9% SLA.

Access Tier

• Hot: Can be used to store frequently accessed data. 
• Cool: Can be used if data access is infrequent. 
• Archive: Can be used to store data that accessed rarely. Only for blob.
  Can't be set on the storage level
  Can set at the blob level 

	Premium Performance	Hot tier	Cool tier	Archive tier
Availability	99.90%	99.90%	99%	Offline
Availability (RA-GRS reads)	N/A	99.99%	99.90%	Offline
Usage charges	Higher storage costs, lower access and transaction cost	Higher storage costs, lower access, and transaction costs	Lower storage costs, higher access, and transaction costs	Lowest storage costs, highest access, and transaction costs
Minimum object size	N/A	N/A	N/A	N/A
Minimum storage duration	N/A	N/A	30 days1	180 days
Latency (Time to first byte)	Single-digit milliseconds	milliseconds	milliseconds	hours2

Azure - Some Basic Concepts of Azure

Some Basic Terminologies 

SAAS: Software as a service
PAAS: Platform as a service
IAAS: Infrastructure as a service


2 O's of Cloud:

1. On-Demand
2. Out Sourced

Resource Group: A logical grouping of resources.
Location: The location on which you create a resource group is metadata of resources, not actual resources.

Ex. You are creating a website for hr management than you can create a resource group HR-Management and keep all the resources you would create for this website in this group.


Deployment: There are 4 options available in VS.

1. Deploy on FTP
2. Deploy in a local directory
3. Deploy on Azure: You can deploy the site directly over Azure.
   If you will use an Azure profile, no need to provide credential every time you would deploy

App Service Editor: This is the online VS Code tool that you can use to edit files at the cloud.

AZR - Availability Zone

(Same like Availability zone in AWS)

Availability Zones is a high-availability offering that protects your applications and data from datacenter failures. Availability Zones are unique physical locations within the Azure region. Each zone is made up of one or more data centers equipped with independent power, cooling, and networking.
To ensure resiliency, there's a minimum of three separate zones in all enabled regions.
The physical separation of Availability Zones within a region protects applications and data from datacenter failures.
Zone-redundant services replicate your applications and data across Availability Zones to protect from single-points-of-failure.
With the Availability Zones, Azure offers industry best 99.99% VM uptime SLA.

Availability zones are subscription-based, which means AZ1 in a specific region of a subscription might be different to AZ1 of the same region in a different subscription.

Availability Zone Support available by Oct 19

Availability set Support
	America				Europe				Asia Pacific
	Central US	East US	East US 2	West US 2	France Central	North Europe	UK South	West Europe	Japan East	Southeast Asia	Australia East
Compute
Linux Virtual Machines	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Windows Virtual Machines	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Virtual Machine Scale Sets	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Azure App Service Environments ILB	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Azure Kubernetes Service	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Storage
Managed Disks	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Zone-redundant Storage	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Networking
Standard IP Address	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Standard Load Balancer	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
VPN Gateway	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
ExpressRoute Gateway	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Application Gateway(V2)	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Azure Firewall	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Databases
Azure Data Explorer	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
SQL Database	Y	Y	Y	Y(Preview)	Y	Y(Preview)	Y	Y	Y	Y	Y
Azure Cache for Redis	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Azure Cosmos DB	Y	Y	Y	Y			Y	Y		Y
Analytics
Event Hubs	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Integration
Service Bus (Premium Tier Only)	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Event Grid	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Identity
Azure AD Domain Services	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y