Pre-requisites Azure Account Azure Data Explorer Cluster Azure Service principal Create a secret Assign API permission for graph to user.read in delegate and application permissions Scope is only to move the meta data from microsoft graph to ADX Azure Machine Learning Workspace Create a compute cluster Code install libraries pip install azure-kusto-ingest …

Optimize parquet file size in 1GB chunks for analytics pre-requisites Azure Account Azure synapse workspace Azure storage account Data set size was 14TB Spark node with large 16vcores and 112GB memory with 2 nodes Synapse Spark First get the data from source from pyspark.sql import SparkSession # Azure storage access info blob_account_name = 'xxxxxxx' # replace with your blob name blob_container_name = 'xxxxxxxx' #…

Yolo V8 in Azure Machine Learning Pre-requisites Azure Machine Learning Service Azure Storage Account Code Create a new Azure Machine Learning Service Workspace Create a notebook Pick the kernel as python with Tensorflow and Pytorch Now clone the repo from github Change conda environment to azureml_py38_TF_PY source active azureml_py38_TF_PY clone the repo git clone https://github.com/ultralytics/ultralytics.git

Curious to learn what ChatGPT thinks about Global Waning Setup Get a ChatGPT account Goal is to understand how good response we get It’s not my intention here to compare or contrast. Just a curious learning to see if ChatGPT can be a personal knowledge system to learn from Learn about Global warming Question: What is happening with Global Warming Answer: Global warming refers…

Use MLFlow with Azure Machine Learning and Automated ML Pre-requisites Azure Subscription Azure Machine Learning Workspace Azure Storage Account Create a data folder upload the data file to the data folder Libraries Install Azure.ai.ml pip install azure.ai.ml Install mlflow pip install mlflow Code Install libraries from azure.identity import DefaultAzureCredential from azure.identity import AzureCliCredential from azure.ai.ml import automl, Input, MLClient

Pre-requisites Open AI Account http://chat.openai.com Get a developer key What is GPT 3.5? Models referred to as "GPT 3.5" GPT-3.5 series is a series of models that was trained on a blend of text and code from before Q4 2021. The following models are in the GPT-3.5 series: code-davinci-002 is a base model, so good…

Example to run Vowpal Wabbit on Azure ML pre-requisites Azure account Azure machine learning Service Steps Create a notebook with python 3.8 with Azure ML as kernel Install vowpal wabbit pip install vowpalwabbit Load the necessary libraries import pandas as pd import sklearn import sklearn.model_selection import sklearn.datasets import vowpalwabbit now load the iris data set into dataframe iris_dataset = sklearn.datasets.load_iris() iris_dataframe…