DataOps for Microsoft Data Platform technologies. https://aka.ms/dataops-repo

Terraform script to deploy almost all Azure Data Services

Threat Detection and Visualization

A data pipeline project build on databricks and azure to demostrate lifecycle of a cloud data project.

Generic Pipelines / Templates for Data Factory / Synapse Pipelines w.r.t Different MSFT Offering Integrations / Use Cases

Azure Container Instances Proxy implemented in Azure Function App (Consumption Plan)

Building a Data Lakehouse using the Medallion architecture.

Project to predict price pass-through, contributing to the company's Revenue area, to predict the ideal SKU value based on various market and competitive variables.

Explore the Tokyo Olympics data journey! We ingested a GitHub CSV into Azure via Data Factory, stored it in Data Lake Storage Gen2, performed transformations in Databricks, conducted advanced analytics in Azure Synapse, and visualized insights in Synapse or Power BI.

This pipeline is an ETL application, which aims to collect data for a few minutes from Binance's open API for cryptocurrencies: BTCUSDT, ADAUSDT, ETHUSDT, BNBUSDT and LTCUSDT.

This repository serves as a sample demonstration of implementing CI/CD practices on Azure Data Factory using Azure DevOps. It provides an example approach to configure a pipeline for continuous integration and continuous deployment (CI/CD) workflows on Azure Data Factory

End-to-End data engineering project with Azure Databricks as cloud service and Tokyo olympic data

Proceso ETL

End-to-end Azure-based data engineering pipeline for CRM analytics using Data Factory, Databricks, Synapse, and Power BI.

No description provided.

End-to-end Azure data engineering pipeline for Spotify data using Azure Data Factory, Databricks, PySpark, and Delta Live Tables with a medallion architecture.

Repository for Azure Data Factory (ADF) Custom Activity to dynamically create and process Azure Analysis Service (AAS) Tabular Model Partitions

Azure Data Factory template to refresh Power BI Dataset

Azure Data Landing Zone - IAC , Config, Code

This Project Extracts supply chain data from csv file having 180k records and more than 40 columns from the Azure Datalake Gen2 storage account and do some dataanalysis with Python(Pandas) to find the top 3 countries and filtered the data for top 3 countries and finally transferred it to 3 files in datalake again by creating ETL pipeline in ADF.

Efetuar o download de arquivos da web com Python. Inserir dados de um dataframe na cloud Azure com Azure SQL Database. Efetuar transformações nos dados com Azure Data Factory.

Interesting_programs_written_in_Python_language

PowerShell Tools

Portfolio Data Analysis and Data Science projects and Data Engineer built using Azure Service, SQL and Python.

This application shows how to use the Azure .NET library to create and execute Azure Data Factory from C# code.

Tokyo Olympic Azure Data Engineering Project

Multi-cloud ETL pipeline using Azure Data Factory to copy CSV files from AWS S3 into Azure Blob Storage, merge them into a single output, and enrich datasets using Data Flow joins. Includes linked service setup, tagging strategy for cost allocation, and optional GitHub integration for CI/CD.

KQL Queries for WWLogs Presentation

Logic App that calls Azure Resource Management REST API to get information about resources in a subscription

rethinkdb jsf 2.3 project