Posts

Showing posts from November, 2021

Streaming Data into Snowflake with AWS DMS

Image
 Amazon Web Service Data Migration Service (AWS DMS) is a cloud-based service where thousands of businesses mobilize data with almost unlimited scale, concurrency, and performance. AWS DMS Snowflake delivers the Data Cloud on AWS in which businesses combine their data in silos, discover and share governed data, and carry out multiple and diverse workloads. Snowflake delivers a single-point seamless experience across various public clouds. With AWS DMS Snowflake , users, customers, and data providers can take their businesses across new frontiers in the Data Cloud.  AWS DMS Snowflake enables you to use the services offered on the AWS Cloud to process a continuous inflow of data and load it into a database of Snowflake. This is done by the following method. Amazon Kines is Firehose is used to deliver the data to Amazon Simple Storage Service (S3). Amazon Simple Notification Service (Amazon SNS) sends notifications when this data is received to Snowflake Snowpipe that in turn loads the da

What is Change Data Capture to A SQL Server Table

Image
 SQL Server Change Data Capture (CDC) is a technology that monitors insert, update and delete activities applicable to a SQL Server table. This results in the details of the changes being available in a relational format that is easy to understand. The metadata and the column information required to apply the changes to a target database are captured for the modified rows and stored in change tables that replicate the column structure of the tracked source tables. Users are provided with table-valued functions that permit methodical access to the change data by consumers.   An example will explain this point. Consider that the data consumer targeted by the CDC technology is an ETL (Extract, Transform, Load) application. In SQL Server Change Data Capture , the ETL application incrementally loads change data from SQL Server source tables to a data mart or a data warehouse. However, even though the source tables in the data warehouse reflect the changes made in the source tables, a techno