Open in app

Sign In

Write

Sign In

Balamurugan Balakreshnan
Balamurugan Balakreshnan

244 Followers

Home

About

Published in MLearning.ai

·12 hours ago

Responsible AI in Predictive Maintenance — Using NASA Turbofan Engine Degradation Dataset — Using sklearn

End to End Train model and perform Responsible AI on NASA Turbofan Engine Degradation Dataset Introduction Using NASA Turbofan Engine Degradation Dataset, we will train a model to predict Remaining Useful Life (RUL) of an engine. Goal is to show how to train the model using automl and perform responsible AI on the model. Get the best run model and perform responsible AI on the model. …

Responsible Ai

10 min read

Responsible Ai

10 min read


Mar 25

Azure Synapse analytics — Data Flow and Synapse Spark — End to End -TPCH data

Using Dataflow and Synapse Spark to analyze data and Spark ML modelling. Using TPCH data Pre-requisites Azure subscription Azure Storage Account Azure Synapse Analytics Load TPCH data I had to limit line items data as it was 150 billion rows. Dataset Rows Orders: 15,000,000,000 Customers: 1,500,000,000 Lineitems: 279,286,998 Goal Use Dataflow to analyze data Use join and create year, month and day columns

Azure Synapse Analytics

4 min read

Azure Synapse analytics — Data Flow and Synapse Spark — End to End -TPCH data
Azure Synapse analytics — Data Flow and Synapse Spark — End to End -TPCH data
Azure Synapse Analytics

4 min read


Published in MLearning.ai

·Mar 18

Azure ML — Python process 20 rows at a time with Azure Open AI

Process large data frame by chunks of 20 Pre-requisites Azure Account Storage account Azure machine learning Azure open ai service Goal Azure Open AI is a service that allows you to use GPT-3 to generate text. But there is limitation on how much we can send at a time. At the time of this document, it was 20 requests/sec. So…

Azureopenai

2 min read

Azure ML — Python process 20 rows at a time with Azure Open AI
Azure ML — Python process 20 rows at a time with Azure Open AI
Azureopenai

2 min read


Published in MLearning.ai

·Mar 11

Process Large text from pdf using Azure Open AI and Azure Form Recognizer

Use Form Recognizer to extract Large text and chunk and summarize Pre-requisites Azure subscription Azure Machine Learning Workspace Document in pdf format Use python Code install azure ai form recognizer pip install azure.ai.formrecognizer import libraries from azure.core.credentials import AzureKeyCredential from azure.ai.formrecognizer import DocumentAnalysisClient import openai import re import requests import sys from num2words import num2words import os import pandas as pd import numpy as np from openai.embeddings_utils import get_embedding, cosine_similarity from transformers…

Form Recognizer

3 min read

Process Large text from pdf using Azure Open AI and Azure Form Recognizer
Process Large text from pdf using Azure Open AI and Azure Form Recognizer
Form Recognizer

3 min read


Published in MLearning.ai

·Mar 4

Summarize PDF document using Azure Open AI using Azure Machine Learning by chunks.

Azure Open AI Summarize in pandas data frame Overview Summarize the data in pandas data frame. Load PDF data into pandas data frame Clean the data. Load all pages into one row per pdf. Using Azure Machine learning Load the pdf in a blob container. Code Pip install pdfreader Load pip reader pip install pdfreader from pdfreader import SimplePDFViewer Import…

Azureopenai

3 min read

Summarize PDF document using Azure Open AI using Azure Machine Learning by chunks.
Summarize PDF document using Azure Open AI using Azure Machine Learning by chunks.
Azureopenai

3 min read


Feb 25

Search using Azure open ai embeddings using Azure Machine learning.

Efficient Embedding Search for Natural Language Queries Introduction use open ai and create embedding Search columns based on embeddings Demo application Restrict 40 rows to azure open ai API install packages pip install tiktoken pip install openai Code Create encoding for gpt2 to get tokens encoding = tiktoken.get_encoding("gpt2") Read the data import pandas as pd df = pd.read_json('sample.json')

Azure Machine Learning

2 min read

Search using Azure open ai embeddings using Azure Machine learning.
Search using Azure open ai embeddings using Azure Machine learning.
Azure Machine Learning

2 min read


Published in MLearning.ai

·Feb 18

Predictive Maintenance using Azure Machine Learning AutoML and Inference using Managed Online endpoint

Using AutoML to predict the failure of a machine Introduction Idea here is to predict the failure of a machine using AutoML AutoML can increase data science process productivity by automating time-consuming, iterative tasks Once the model is developed, we will deploy the model as a web service For deployment we are using Managed Online endpoint, Fully managed service that…

Predictive Maintenance

5 min read

Predictive Maintenance using Azure Machine Learning AutoML and Inference using Managed Online…
Predictive Maintenance using Azure Machine Learning AutoML and Inference using Managed Online…
Predictive Maintenance

5 min read


Published in MLearning.ai

·Feb 11

Azure Machine learning — Ray parallel processing

Execute Ray parallel processing in Azure Machine Learning with single node CPU Prerequisites Azure subscription Azure Machine Learning Workspace Azure Storage Account Install Ray library Code install ray libraries pip install -qU "ray[tune]" lightgbm_ray install pyarrow specific version for the code to work pip install pyarrow==6.0.1 Restart the kernel import libraries from typing import Tuple import ray from ray.train.batch_predictor import BatchPredictor from ray.train.lightgbm import LightGBMPredictor from ray.data.preprocessors.chain…

Azure Machine Learning

2 min read

Azure Machine learning — Ray parallel processing
Azure Machine learning — Ray parallel processing
Azure Machine Learning

2 min read


Published in MLearning.ai

·Feb 6

Azure Open AI with Power Apps

Let’s build a Power App to use Azure Open AI for various use cases. What’s needed. Register for Azure Open AI — https://learn.microsoft.com/en-us/azure/cognitive-services/openai/overview Once got approved create a azure open ai resource in Azure portal Select region as East US At the time of writing this article davinci-003 is only available in East US Create a deployment inside the resource Create a Power App

OpenAI

6 min read

Azure Open AI with Power Apps
Azure Open AI with Power Apps
OpenAI

6 min read


Published in MLearning.ai

·Jan 28

Move Microsoft Graph metadata to Azure Data Explorer using pandas dataframe

Pre-requisites Azure Account Azure Data Explorer Cluster Azure Service principal Create a secret Assign API permission for graph to user.read in delegate and application permissions Scope is only to move the meta data from microsoft graph to ADX Azure Machine Learning Workspace Create a compute cluster Code install libraries pip install azure-kusto-ingest …

Microsoft Graph Api

2 min read

Move Microsoft Graph metadata to Azure Data Explorer using pandas dataframe
Move Microsoft Graph metadata to Azure Data Explorer using pandas dataframe
Microsoft Graph Api

2 min read

Balamurugan Balakreshnan

Balamurugan Balakreshnan

244 Followers

https://balakreshnan.github.io/

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech