Engineering Excellence

Engineering Excellence

Having solid engineering process and tools in place help to improve the agility of the system. Here is the high-level flow recommended by our VP.

While building the solution we need to see how we can bring value, MVP instead of build parts which is not adding value initially.

AI Tools

• https://app.mindpal.space
• https://riverside.fm
• Gammas - Gamma
• https://fathom.video 

Microsoft copilot Agent

 In Copilot Chat https://m365.cloud.microsoft/chat

Click on All Agents on side bar

Copilot Studio

Provide the details and you are agent is ready

Teams PowerShell

 The PowerShell helps to get quick meta data around Teams.

Install-Module -Name MicrosoftTeams
Connect-MicrosoftTeams
Get-TeamAllChannel -GroupId   | select DisplayName | clip

Azure Machine Learning

Azure Machine Learning is the home for Data Scientists to run experiments.  But the language and constructs used in Microsoft Azure ML will be confusing in the first go.  This article helps to clear some fundamentals.

The below words are confusing, and the UI is not that great.

• Pipeline
• You can create a pipeline object and execute in an experiment.
• Pipeline Job
• Experiment: Pipeline jobs are grouped into experiments to organize job history. You can set the experiment for every pipeline job.
• Pipeline Endpoint
• Can have multiple Pipeline but only one could be the default.
• Published Pipeline

Running a script - Experiment Job Type: Command 

from azureml.core import Workspace, Experiment, Environment, ScriptRunConfig

ws = Workspace.from_config()

experiment = Experiment(workspace=ws, name='day1-experiment-try1')

config = ScriptRunConfig(source_directory='./', script='Hello.py', compute_target='gurupc')

run = experiment.submit(config)

run.log('mymetric', 1)

run.wait_for_completion(show_output=True, wait_post_processing=True)

aml_url = run.get_portal_url()

print(aml_url)

Running a Pipeline - Experiment Job Type: Pipeline

https://github.com/guvijaya/TipsAndTricks/blob/master/Azure%20Machine%20Learning/AML%20Pipelines%20Intro.py

External Tables in SQL

CREATE DATABASE SCOPED CREDENTIAL sql_user_bob
WITH
     IDENTITY = 'sql_user_bob',
     SECRET = 'dtgf#45322@d343' ;
go

CREATE EXTERNAL DATA SOURCE [NameExternalDB]
WITH (TYPE = RDBMS, LOCATION = N'ExternalSerer.database.windows.net',
CREDENTIAL = [datahub_reader], DATABASE_NAME = N'ExternalDB')
GO

CREATE EXTERNAL TABLE [dbo].[dimUsers]
(
[UserId] [int] NULL,
....
)
WITH (DATA_SOURCE = [NameExternalDB] ,SCHEMA_NAME = N'common',OBJECT_NAME = N'dimUsers')
GO

History tracking for SQL

Add history tracking to the existing table

alter table tbl_event2
add SysStartTime datetime2 generated always as row start not null default getutcdate(),
    SysEndTime   datetime2 generated always as row end   not null default convert(datetime2, '9999-12-31 23:59:59.9999999'),
    period for system_time (SysStartTime, SysEndTime);

alter table tbl_event2
  set (system_versioning = on (HISTORY_TABLE = dbo.tbl_event2_history));