Copilot Is Broken Until You Do THIS

1
00:00:00,000 --> 00:00:02,440
You think Copilot is smart because it speaks confidently.

2
00:00:02,440 --> 00:00:04,520
The truth, it's a pleasant generalist that

3
00:00:04,520 --> 00:00:07,720
stalls the second your question touches your company's rules.

4
00:00:07,720 --> 00:00:11,000
You paid for a digital butler, you got an intern with Wi-Fi.

5
00:00:11,000 --> 00:00:12,120
Here's the fix.

6
00:00:12,120 --> 00:00:16,160
Bring your custom engine agent into Microsoft 365 Copilot chat.

7
00:00:16,160 --> 00:00:18,520
I'll show you the exact two-minute manifest tweak that

8
00:00:18,520 --> 00:00:21,600
unlocks it plus the guardrails to keep security teams calm.

9
00:00:21,600 --> 00:00:23,320
We'll test it against real policies,

10
00:00:23,320 --> 00:00:26,360
data loss prevention, regional SOPs and compliance,

11
00:00:26,360 --> 00:00:28,880
so you see speed, accuracy, and fewer hallucinations

12
00:00:28,880 --> 00:00:29,880
immediately.

13
00:00:29,880 --> 00:00:32,960
Let's upgrade the intern into a specialist.

14
00:00:32,960 --> 00:00:36,080
The problem, where default Copilot fails in the real world.

15
00:00:36,080 --> 00:00:36,800
Picture this.

16
00:00:36,800 --> 00:00:39,160
You ask Copilot, can I share a customer spreadsheet

17
00:00:39,160 --> 00:00:40,600
with an external vendor?

18
00:00:40,600 --> 00:00:43,040
It gives a polished paragraph on best practices,

19
00:00:43,040 --> 00:00:45,800
sprinkles in generic Microsoft 365 guidance,

20
00:00:45,800 --> 00:00:48,120
and spoiler alert says nothing about your tenant's

21
00:00:48,120 --> 00:00:49,800
data loss prevention policy.

22
00:00:49,800 --> 00:00:51,400
The average user notes shares the file

23
00:00:51,400 --> 00:00:52,600
and triggers a policy alert.

24
00:00:52,600 --> 00:00:54,840
Yes, they require restarts, pause it,

25
00:00:54,840 --> 00:00:56,960
because Microsoft is not performing magic tricks.

26
00:00:56,960 --> 00:01:00,120
Copilot cannot intuit your internal rules unless you inject them.

27
00:01:00,120 --> 00:01:01,760
Now compare that to what you actually need.

28
00:01:01,760 --> 00:01:04,120
Your security team defined a DLP policy

29
00:01:04,120 --> 00:01:07,240
with very specific conditions, customer PII over a threshold,

30
00:01:07,240 --> 00:01:09,520
financial identifiers, sharing destinations

31
00:01:09,520 --> 00:01:13,080
limited by region and exceptions for a named project code.

32
00:01:13,080 --> 00:01:15,320
Default Copilot doesn't know your exception list.

33
00:01:15,320 --> 00:01:17,320
It doesn't know your project Orion carve out.

34
00:01:17,320 --> 00:01:18,840
It can describe DLP in theory,

35
00:01:18,840 --> 00:01:20,520
it can't apply your DLP in practice.

36
00:01:20,520 --> 00:01:22,200
That gap is where incidents happen.

37
00:01:22,200 --> 00:01:24,880
Let's run a second scenario, regional SOPs.

38
00:01:24,880 --> 00:01:26,800
You ask, what's the approved escalation

39
00:01:26,800 --> 00:01:30,360
path for a servo2 outage in EMIRA after 6pm?

40
00:01:30,360 --> 00:01:33,640
Default Copilot happily cites generic ITIL principles,

41
00:01:33,640 --> 00:01:37,040
then recommends contacting the on-call team, fascinating,

42
00:01:37,040 --> 00:01:37,960
which team?

43
00:01:37,960 --> 00:01:41,320
In your org, EMIRA after hours roots to a regional vendor,

44
00:01:41,320 --> 00:01:44,320
not the core SRE rotation, there's a name distribution list,

45
00:01:44,320 --> 00:01:47,320
a paging policy, and a 20 minute acknowledgement SLA tied

46
00:01:47,320 --> 00:01:48,440
to a penalty clause.

47
00:01:48,440 --> 00:01:51,280
The generic answer is not only useless, it's dangerous.

48
00:01:51,280 --> 00:01:54,000
The clock is ticking and you're emailing the wrong queue.

49
00:01:54,000 --> 00:01:56,720
Third scenario, compliance.

50
00:01:56,720 --> 00:01:58,840
Are we allowed to email HIPAA related updates

51
00:01:58,840 --> 00:02:01,120
to patients using Outlook campaigns?

52
00:02:01,120 --> 00:02:03,840
Default Copilot will serenade you with reassuring words

53
00:02:03,840 --> 00:02:05,880
about encryption and secure handling.

54
00:02:05,880 --> 00:02:07,880
It will not recall your legal department's memo

55
00:02:07,880 --> 00:02:09,880
that mandates patient notices must go

56
00:02:09,880 --> 00:02:12,520
through the patient portal with multifactor re-auth,

57
00:02:12,520 --> 00:02:13,720
not bulk email.

58
00:02:13,720 --> 00:02:16,600
Again, general knowledge, masquerading, as authority,

59
00:02:16,600 --> 00:02:18,720
if you wanted a TED talk, you wouldn't be here.

60
00:02:18,720 --> 00:02:19,840
Why does this keep happening?

61
00:02:19,840 --> 00:02:22,240
Because out of the box, Copilot is optimized for breadth,

62
00:02:22,240 --> 00:02:23,040
not your depth.

63
00:02:23,040 --> 00:02:25,480
It's grounded by Microsoft Graph and Public Knowledge,

64
00:02:25,480 --> 00:02:27,400
which is the right default for most people,

65
00:02:27,400 --> 00:02:29,600
but the wrong default for enterprise precision.

66
00:02:29,600 --> 00:02:31,320
Think of Microsoft Graph as the skeleton,

67
00:02:31,320 --> 00:02:33,120
structure without your muscle memory.

68
00:02:33,120 --> 00:02:35,400
Your policies, SOPs and compliance interpretations

69
00:02:35,400 --> 00:02:36,600
are the tendons and nerves.

70
00:02:36,600 --> 00:02:39,120
Without them, the system moves, but clumsily.

71
00:02:39,120 --> 00:02:41,800
And the thing most people miss, hallucinations feel helpful.

72
00:02:41,800 --> 00:02:43,960
The language is coherent, the verbs are confident.

73
00:02:43,960 --> 00:02:45,080
So users trusted.

74
00:02:45,080 --> 00:02:47,920
The problem isn't that Copilot is malicious, it's oblivious.

75
00:02:47,920 --> 00:02:49,280
It doesn't know your tenant controls,

76
00:02:49,280 --> 00:02:51,040
your RRI filters, your data scopes,

77
00:02:51,040 --> 00:02:53,240
or the political landmines inside your company.

78
00:02:53,240 --> 00:02:54,880
It's not just missing a database,

79
00:02:54,880 --> 00:02:57,040
it's missing the spine of your governance.

80
00:02:57,040 --> 00:02:59,120
I watched a team waste an hour last week

81
00:02:59,120 --> 00:03:01,280
chasing a recommended approval path

82
00:03:01,280 --> 00:03:03,720
Copilot fabricated from adjacent truths.

83
00:03:03,720 --> 00:03:06,040
They had the right documents, buried in SharePoint,

84
00:03:06,040 --> 00:03:08,600
written in legalese, updated quarterly.

85
00:03:08,600 --> 00:03:10,560
Default Copilot couldn't follow the breadcrumbs

86
00:03:10,560 --> 00:03:12,440
because it didn't know which crumbs mattered.

87
00:03:12,440 --> 00:03:15,320
When this finally clicks, you stop expecting the generalist

88
00:03:15,320 --> 00:03:16,560
to behave like a specialist

89
00:03:16,560 --> 00:03:18,400
and start wiring in the specialist.

90
00:03:18,400 --> 00:03:19,920
Here's the brutal audit.

91
00:03:19,920 --> 00:03:22,520
Anywhere your question depends on local nuance.

92
00:03:22,520 --> 00:03:27,160
Exception codes, region-specific SOPs, conditional approvals,

93
00:03:27,160 --> 00:03:30,080
trade-restricted SKUs, retention schedules,

94
00:03:30,080 --> 00:03:32,080
or compensation rules.

95
00:03:32,080 --> 00:03:35,360
Default Copilot will either generalize or hallucinate.

96
00:03:35,360 --> 00:03:38,600
Both waste time, one creates risk.

97
00:03:38,600 --> 00:03:41,080
The truth, until you bring your own engine,

98
00:03:41,080 --> 00:03:43,680
your retrieval, your actions, your policies,

99
00:03:43,680 --> 00:03:45,800
Copilot will skate on the surface.

100
00:03:45,800 --> 00:03:48,160
You need an agent that can pass your policy text,

101
00:03:48,160 --> 00:03:50,400
apply your decision tree, respect your tenant boundaries,

102
00:03:50,400 --> 00:03:52,560
and answer with citations from your corpus.

103
00:03:52,560 --> 00:03:54,280
Otherwise, you're paying for premium tires

104
00:03:54,280 --> 00:03:55,640
and driving on ice.

105
00:03:55,640 --> 00:03:57,760
The fix is not a motivational speech.

106
00:03:57,760 --> 00:04:00,360
It's a two-minute manifest change that invites your specialist

107
00:04:00,360 --> 00:04:01,680
into Copilot chat.

108
00:04:01,680 --> 00:04:04,320
And yes, we're doing that next on the better method,

109
00:04:04,320 --> 00:04:06,400
custom engine agents as specialists.

110
00:04:06,400 --> 00:04:08,640
Enter the specialist, a custom engine agent.

111
00:04:08,640 --> 00:04:10,800
Not a cute plug-in, not a glorified prompt,

112
00:04:10,800 --> 00:04:13,760
a programmable colleague that speaks your organization's dialect

113
00:04:13,760 --> 00:04:15,240
because you taught it the language.

114
00:04:15,240 --> 00:04:18,240
The truth, this is how you stop Copilot from freelancing

115
00:04:18,240 --> 00:04:20,160
and started practicing policy.

116
00:04:20,160 --> 00:04:21,120
What is it precisely?

117
00:04:21,120 --> 00:04:22,800
A custom engine agent is an agent you

118
00:04:22,800 --> 00:04:25,640
build with real orchestration, semantic kernel or lang chain,

119
00:04:25,640 --> 00:04:27,440
wired into your data and your rules.

120
00:04:27,440 --> 00:04:30,480
It plans, calls, tools, keeps state,

121
00:04:30,480 --> 00:04:32,760
and returns answers grounded in your corpus.

122
00:04:32,760 --> 00:04:35,400
Think of default Copilot as the operating room.

123
00:04:35,400 --> 00:04:38,400
Your custom agent is the surgeon you credential to work there.

124
00:04:38,400 --> 00:04:39,760
No credentials, no scalpel.

125
00:04:39,760 --> 00:04:40,840
Why this works is simple.

126
00:04:40,840 --> 00:04:42,800
The thing most people miss is that authority

127
00:04:42,800 --> 00:04:45,960
comes from three ingredients, retrieval, actions, and guardrails.

128
00:04:45,960 --> 00:04:48,960
Retrieval gives the agent memory beyond vibes.

129
00:04:48,960 --> 00:04:50,640
Actions let it do real work.

130
00:04:50,640 --> 00:04:54,080
Hit an API, query a system, generate a report.

131
00:04:54,080 --> 00:04:56,880
Guardrails keep it inside the lines, tenant controls,

132
00:04:56,880 --> 00:04:59,320
responsible AI filters, and data scopes,

133
00:04:59,320 --> 00:05:01,480
so it never wanders into places it shouldn't.

134
00:05:01,480 --> 00:05:03,680
Let me show you exactly how the specialist is assembled.

135
00:05:03,680 --> 00:05:04,760
First, retrieval.

136
00:05:04,760 --> 00:05:07,280
You connect Azure AI search to your governed content,

137
00:05:07,280 --> 00:05:10,800
SOPs, policy pages, legal memos, DLP rule descriptions,

138
00:05:10,800 --> 00:05:11,640
and change logs.

139
00:05:11,640 --> 00:05:13,960
Use hybrid search, vector plus keyword,

140
00:05:13,960 --> 00:05:16,000
because your users ask messy questions

141
00:05:16,000 --> 00:05:18,160
and your documents use precise terms.

142
00:05:18,160 --> 00:05:21,400
The agent decomposes the query, pulls targeted chunks,

143
00:05:21,400 --> 00:05:23,200
and returns answers with citations.

144
00:05:23,200 --> 00:05:24,760
If you remember nothing else, citations

145
00:05:24,760 --> 00:05:26,920
are the antidote to hallucinations.

146
00:05:26,920 --> 00:05:29,960
Second, reasoning and orchestration.

147
00:05:29,960 --> 00:05:34,680
With semantic kernel, you define plug-ins, skills that encapsulate tasks.

148
00:05:34,680 --> 00:05:37,720
Get DLP exception list, resolve regional escalation,

149
00:05:37,720 --> 00:05:39,320
check retention schedule.

150
00:05:39,320 --> 00:05:41,720
The planner can chain these based on the question.

151
00:05:41,720 --> 00:05:44,160
Lang chain does the same with tools and chains

152
00:05:44,160 --> 00:05:46,440
if you prefer more open architecture.

153
00:05:46,440 --> 00:05:50,040
Either way, you get a repeatable flow instead of improv theater.

154
00:05:50,040 --> 00:05:51,600
Third, actions.

155
00:05:51,600 --> 00:05:53,600
Retrieval is nice, action is value.

156
00:05:53,600 --> 00:05:57,160
Your agent can call internal APIs to validate an exception code,

157
00:05:57,160 --> 00:05:59,200
check who's on call in a mere tonight,

158
00:05:59,200 --> 00:06:02,400
or verify that a patient notification channel is approved.

159
00:06:02,400 --> 00:06:06,960
The shortcut nobody teaches, tiny boring actions drive massive accuracy.

160
00:06:06,960 --> 00:06:09,240
A single validate project code tool

161
00:06:09,240 --> 00:06:13,080
can collapse paragraphs of uncertainty into a one-line answer.

162
00:06:13,080 --> 00:06:15,000
Now guardrails, before we continue,

163
00:06:15,000 --> 00:06:19,200
you need to understand this, or you'll get blocked by security in 20 minutes.

164
00:06:19,200 --> 00:06:21,320
Tenant controls determine where the agent can run

165
00:06:21,320 --> 00:06:22,880
and what data it can touch.

166
00:06:22,880 --> 00:06:25,400
Data scopes, restrict indices, and repositories

167
00:06:25,400 --> 00:06:27,960
by department, region, and sensitivity.

168
00:06:27,960 --> 00:06:30,760
Our eye filters handle content safety and prompt hygiene.

169
00:06:30,760 --> 00:06:33,040
Compare that to letting a new hire roam your file shares

170
00:06:33,040 --> 00:06:36,160
because they ask nicely, no, you scope them, you monitor them,

171
00:06:36,160 --> 00:06:37,240
you log them.

172
00:06:37,240 --> 00:06:37,960
Same here.

173
00:06:37,960 --> 00:06:40,720
Observability and access boundaries are not optional.

174
00:06:40,720 --> 00:06:43,000
Once you nail that, everything else clicks.

175
00:06:43,000 --> 00:06:44,640
The agent doesn't guess at DLP.

176
00:06:44,640 --> 00:06:48,080
It reads your DLP artifact, checks the exception list, and returns.

177
00:06:48,080 --> 00:06:50,280
No, external sharing is blocked for PII

178
00:06:50,280 --> 00:06:52,520
unless Project Orion Exemption is active.

179
00:06:52,520 --> 00:06:53,800
Open a request here.

180
00:06:53,800 --> 00:06:57,920
For regional SOPs, it uses time, region, and severity as parameters,

181
00:06:57,920 --> 00:07:02,840
then cites the exact page that names the after-hours vendor and the 20-minute SLA.

182
00:07:02,840 --> 00:07:06,200
For HIPPA notices, it calls the policy endpoint and replies,

183
00:07:06,200 --> 00:07:10,360
use the patient portal, MFA-inforced, bulk email is prohibited,

184
00:07:10,360 --> 00:07:13,200
with the legal memo link, Confidence with Receipts.

185
00:07:13,200 --> 00:07:15,480
The game changer nobody talks about is ownership.

186
00:07:15,480 --> 00:07:17,960
With a custom engine agent, you control the model choice,

187
00:07:17,960 --> 00:07:20,680
the retrieval settings, the action contracts, and the logging.

188
00:07:20,680 --> 00:07:24,360
You can swap GPT variants, tune chunking, re-rank results,

189
00:07:24,360 --> 00:07:25,520
and instrument every step.

190
00:07:25,520 --> 00:07:28,360
That's how you get speed, accuracy, and reduced hallucinations

191
00:07:28,360 --> 00:07:30,000
without praying to the demo gods.

192
00:07:30,000 --> 00:07:32,520
And yes, you still want co-pilot's UX and discovery.

193
00:07:32,520 --> 00:07:33,520
That's where we're headed.

194
00:07:33,520 --> 00:07:36,840
The two-minute manifest tweak that gives your specialist a seat in co-pilot chat,

195
00:07:36,840 --> 00:07:41,000
so users ask one assistant and the right brain answers, you build a surgeon,

196
00:07:41,000 --> 00:07:43,520
then you badge it into the operating room.

197
00:07:43,520 --> 00:07:46,160
The two-minute manifest tweak that unlocks co-pilot chat.

198
00:07:46,160 --> 00:07:48,080
Okay, surgeon built, now we badge them in.

199
00:07:48,080 --> 00:07:50,080
This is the two-minute tweak everyone skips,

200
00:07:50,080 --> 00:07:52,840
then wonders why their agent only shows up in teams.

201
00:07:52,840 --> 00:07:56,440
It's not magic, it's the manifest, the apps passport, update the passport,

202
00:07:56,440 --> 00:07:58,800
the border agent waves you through, refuse,

203
00:07:58,800 --> 00:08:01,160
and you stay in the lobby with the average users.

204
00:08:01,160 --> 00:08:02,720
Step one, bump the schema.

205
00:08:02,720 --> 00:08:05,280
Open your app manifest JSON, find the schema reference,

206
00:08:05,280 --> 00:08:09,200
and update it to version 1.22, not 120, not latest.

207
00:08:09,200 --> 00:08:13,600
1.22, the truth, co-pilot chat looks for capabilities that only exist in that schema.

208
00:08:13,600 --> 00:08:17,040
If the schema's older, your agent can be perfect and still get ignored,

209
00:08:17,040 --> 00:08:19,040
you called it the cloud.

210
00:08:19,040 --> 00:08:21,880
Incorrect, it's a distributed rules engine,

211
00:08:21,880 --> 00:08:24,640
and the schema is how you speak those rules.

212
00:08:24,640 --> 00:08:27,880
Step two, tell co-pilot you are, in fact, a co-pilot agent.

213
00:08:27,880 --> 00:08:31,360
In the capabilities section for your bot at the co-pilot agent tags,

214
00:08:31,360 --> 00:08:34,360
you'll see a capabilities array include co-pilot agents,

215
00:08:34,360 --> 00:08:38,440
and if you're bringing your own orchestration, declare custom engine agents.

216
00:08:38,440 --> 00:08:39,680
This is the identity badge.

217
00:08:39,680 --> 00:08:43,600
Without it, co-pilot assumes you're just here for team's small talk and adaptive cards.

218
00:08:43,600 --> 00:08:46,920
With it, co-pilot routes relevant chats to your specialist.

219
00:08:46,920 --> 00:08:49,400
Inside the same experience, users already live in.

220
00:08:49,400 --> 00:08:51,160
Step three, conversation starters.

221
00:08:51,160 --> 00:08:54,400
Still in 1.22, you can define up to 12 localized prompts.

222
00:08:54,400 --> 00:08:55,600
Use them.

223
00:08:55,600 --> 00:08:57,840
Ask about DLP sharing exceptions.

224
00:08:57,840 --> 00:09:00,120
Check Emia after our escalation.

225
00:09:00,120 --> 00:09:02,440
Verify HIPAA notification channel.

226
00:09:02,440 --> 00:09:03,760
These aren't decorative.

227
00:09:03,760 --> 00:09:07,840
They are discovery beacons that teach users what the agent is good at.

228
00:09:07,840 --> 00:09:10,920
Fewer aimless prompts, fewer hallucinations higher accuracy.

229
00:09:10,920 --> 00:09:14,160
Yes, this is how you train humans without making them read a manual.

230
00:09:14,160 --> 00:09:16,640
Step four, actions.

231
00:09:16,640 --> 00:09:21,240
The updated schema lets you enumerate richer actions your agent can invoke.

232
00:09:21,240 --> 00:09:24,080
Your internal APIs, your policy lookup endpoints,

233
00:09:24,080 --> 00:09:26,960
declare them cleanly with parameters and descriptions.

234
00:09:26,960 --> 00:09:28,640
Co-pilot uses this to plan.

235
00:09:28,640 --> 00:09:30,280
If you leave it blank, it guesses.

236
00:09:30,280 --> 00:09:32,480
When software guesses, people file tickets.

237
00:09:32,480 --> 00:09:33,680
Now, packaging.

238
00:09:33,680 --> 00:09:37,600
Zip the app with the updated manifest and deploy it the way you normally do.

239
00:09:37,600 --> 00:09:39,920
Teams admin center or your app catalog.

240
00:09:39,920 --> 00:09:41,760
Here's the test most people miss.

241
00:09:41,760 --> 00:09:45,240
In co-pilot chat, start typing a scenario your agent owns.

242
00:09:45,240 --> 00:09:49,840
You should see your agent offered as a participant or invoked in line based on triggers.

243
00:09:49,840 --> 00:09:54,480
If you don't, you either forgot the tags, bot ID mismatch, or your tenant controls are blocking

244
00:09:54,480 --> 00:09:55,480
the channel.

245
00:09:55,480 --> 00:09:57,480
Yes, tenant controls exist.

246
00:09:57,480 --> 00:09:59,080
No you can't wish them away.

247
00:09:59,080 --> 00:10:00,400
Verification is simple.

248
00:10:00,400 --> 00:10:02,280
Ask the three scenarios we said earlier.

249
00:10:02,280 --> 00:10:06,240
You want answers with citations from your corpus, not generic platitudes.

250
00:10:06,240 --> 00:10:07,480
Watch for latency.

251
00:10:07,480 --> 00:10:12,680
Schema 1.22 plus the right capabilities cuts routing friction so responses land faster.

252
00:10:12,680 --> 00:10:17,720
If it still feels sluggish, that's your retrieval settings, not the manifest.

253
00:10:17,720 --> 00:10:19,360
Common mistakes to avoid.

254
00:10:19,360 --> 00:10:23,840
Leaving the app scope to teams only, forgetting localization, omitting security attestations

255
00:10:23,840 --> 00:10:26,080
and deploying to the wrong environment.

256
00:10:26,080 --> 00:10:28,640
Also don't add every capability because you can.

257
00:10:28,640 --> 00:10:29,880
Precision beats maximalism.

258
00:10:29,880 --> 00:10:32,760
Declare what you actually support and log everything.

259
00:10:32,760 --> 00:10:36,960
Once you nail this, your specialist sits inside co-pilot chat like it always belonged

260
00:10:36,960 --> 00:10:37,960
there.

261
00:10:37,960 --> 00:10:39,480
Same UI, different brain.

262
00:10:39,480 --> 00:10:44,440
And yes, we're about to wrap guard rails around it so your security team doesn't faint.

263
00:10:44,440 --> 00:10:45,440
Implementation.

264
00:10:45,440 --> 00:10:48,200
Build, deploy, verify, end to end.

265
00:10:48,200 --> 00:10:49,200
Let's do the end to end.

266
00:10:49,200 --> 00:10:50,200
No hand waving.

267
00:10:50,200 --> 00:10:55,080
You'll build the specialist with real retrieval and actions, package it with the 1.22 manifest,

268
00:10:55,080 --> 00:10:59,360
deploy to your tenant and verify against DLP, regional SOPs and compliance.

269
00:10:59,360 --> 00:11:02,120
You'll see speed, accuracy, and fewer hallucinations.

270
00:11:02,120 --> 00:11:04,280
Not because we prayed, but because we engineered.

271
00:11:04,280 --> 00:11:05,280
Build.

272
00:11:05,280 --> 00:11:06,280
Part 1.

273
00:11:06,280 --> 00:11:07,280
Build.

274
00:11:07,280 --> 00:11:10,360
Create an Azure AI search index for your governed content.

275
00:11:10,360 --> 00:11:16,240
DLP policy docs, exception procedures, regional SOPs, legal memos, and change logs, use hybrid

276
00:11:16,240 --> 00:11:19,200
search, vector plus keyword, and enable semantic ranking.

277
00:11:19,200 --> 00:11:23,520
Chunk documents by headings with overlaps of context survives boundaries, store citations

278
00:11:23,520 --> 00:11:28,420
with persistent IDs, configure enrichment to extract key entities like project codes,

279
00:11:28,420 --> 00:11:33,480
regions, severities and distribution lists because spoiler alert your users ask in those terms.

280
00:11:33,480 --> 00:11:37,440
Index updates, automate them, a stale index is how yesterday's policy becomes today's

281
00:11:37,440 --> 00:11:38,440
incident.

282
00:11:38,440 --> 00:11:39,440
Build.

283
00:11:39,440 --> 00:11:40,440
Part 2.

284
00:11:40,440 --> 00:11:41,840
Orchestration that doesn't improvise.

285
00:11:41,840 --> 00:11:46,480
In semantic kernel, define plugins, get DLP exceptions, project code, resolve escalation,

286
00:11:46,480 --> 00:11:50,320
region severity, local time, check notice channel policy area.

287
00:11:50,320 --> 00:11:56,040
Give each plugin narrow testable behavior and return structured outputs, objects, not essays.

288
00:11:56,040 --> 00:12:00,240
Wrap as your AI search as a retriever with agente query planning turned on.

289
00:12:00,240 --> 00:12:05,480
Decompose complex prompts into subquaries, fan out, gather top-k, re-rank, synthesize

290
00:12:05,480 --> 00:12:08,040
with citations, set a conservative temperature.

291
00:12:08,040 --> 00:12:09,560
You want answers, not poetry.

292
00:12:09,560 --> 00:12:14,120
If you prefer lang chain tools and a graph-based agent do the same job, just apply the same discipline,

293
00:12:14,120 --> 00:12:17,000
typed outputs, defensive defaults, exhaustive logging.

294
00:12:17,000 --> 00:12:18,000
Build.

295
00:12:18,000 --> 00:12:19,000
Part 3.

296
00:12:19,000 --> 00:12:21,080
Actions that collapse uncertainty.

297
00:12:21,080 --> 00:12:22,720
Connect to your internal APIs.

298
00:12:22,720 --> 00:12:27,800
The DLP exception registry, the on-call schedule service and the compliance policy endpoint,

299
00:12:27,800 --> 00:12:31,960
protect them with managed identities and scope permissions to the agent service principle.

300
00:12:31,960 --> 00:12:35,680
Add a tiny boring tool, validate project code, project code.

301
00:12:35,680 --> 00:12:37,720
It returns true or false and a reason.

302
00:12:37,720 --> 00:12:41,560
That single boolean cuts paragraphs of hedging into a definitive sentence.

303
00:12:41,560 --> 00:12:45,600
The thing most people miss is that one or two high signal calls beat five pages of speculative

304
00:12:45,600 --> 00:12:50,280
reasoning, guardrails because you like your job, enable tenant controls, so the app is

305
00:12:50,280 --> 00:12:53,080
available only in approved environments.

306
00:12:53,080 --> 00:12:55,000
Restrict data scopes.

307
00:12:55,000 --> 00:12:59,240
The agent's retriever touches only the indices it needs, no wandering through HR files

308
00:12:59,240 --> 00:13:01,440
because someone typed benefits.

309
00:13:01,440 --> 00:13:05,720
Turn on Rye filters appropriate for your content, profanity, sensitive terms and jailbreak

310
00:13:05,720 --> 00:13:06,720
defense.

311
00:13:06,720 --> 00:13:10,960
Instrument everything, trace IDs across retrieval, tools and synthesis, log prompts and responses

312
00:13:10,960 --> 00:13:14,240
with reduction, emit metrics for latency and hit rates.

313
00:13:14,240 --> 00:13:18,720
If security asks for an audit trail and you shrug, you will lose this program in the meeting.

314
00:13:18,720 --> 00:13:23,400
Now package, update the manifest schema to 1.22 at copilot agents and if you're bringing

315
00:13:23,400 --> 00:13:27,200
your own orchestration, custom engine agents and capabilities.

316
00:13:27,200 --> 00:13:33,200
Define conversation status, 12 max localized, declare actions with parameters and descriptions.

317
00:13:33,200 --> 00:13:35,640
Include security attestations if your org requires them.

318
00:13:35,640 --> 00:13:37,560
Map your bot ID correctly.

319
00:13:37,560 --> 00:13:40,920
Production is not your devgoid, zip the app with icon privacy and terms.

320
00:13:40,920 --> 00:13:41,920
This is not busy work.

321
00:13:41,920 --> 00:13:46,160
This is the difference between visible in copilot and why can't I find it.

322
00:13:46,160 --> 00:13:49,160
Deploy via team's admin center or your app catalog.

323
00:13:49,160 --> 00:13:53,960
Go up to a pilot group first, operations and compliance champions who will actually use it.

324
00:13:53,960 --> 00:13:56,640
Verify tenant-wide settings for copilot extensibility.

325
00:13:56,640 --> 00:14:00,480
If your admin disabled external plugins and agent channels, your specialist will sit outside

326
00:14:00,480 --> 00:14:03,440
like a locked out surgeon, install for pilot users.

327
00:14:03,440 --> 00:14:07,840
In copilot chat, your agent should appear as an available participant and as inline invocation

328
00:14:07,840 --> 00:14:08,920
on relevant prompts.

329
00:14:08,920 --> 00:14:13,760
If it doesn't, check capabilities, app permission policies and bot messaging endpoints.

330
00:14:13,760 --> 00:14:16,640
Copilot doesn't guess your intent, it reads your manifest.

331
00:14:16,640 --> 00:14:18,040
Time for the three proof tests.

332
00:14:18,040 --> 00:14:23,000
One, DLP, ask, can I share this customer spreadsheet with our external vendor?

333
00:14:23,000 --> 00:14:25,080
Provide a fake path and a project code.

334
00:14:25,080 --> 00:14:26,280
Expect, no.

335
00:14:26,280 --> 00:14:30,480
External sharing of PII is blocked unless project Orion exemption is active.

336
00:14:30,480 --> 00:14:32,000
Status, inactive.

337
00:14:32,000 --> 00:14:33,200
Request exemption here.

338
00:14:33,200 --> 00:14:36,960
With a link and citations to the DLP policy and exception runbook.

339
00:14:36,960 --> 00:14:38,240
That is accuracy plus action.

340
00:14:38,240 --> 00:14:42,160
If it hedges, your validate project code tool is missing or your retriever couldn't find

341
00:14:42,160 --> 00:14:43,160
the exception policy.

342
00:14:43,160 --> 00:14:44,960
Two, regional SO piece.

343
00:14:44,960 --> 00:14:46,720
It's 1910 CET, save two in MIR.

344
00:14:46,720 --> 00:14:47,960
Who do I page?

345
00:14:47,960 --> 00:14:51,800
To the vendor's on call name, the escalation channel and the 20 minute SLA with citations

346
00:14:51,800 --> 00:14:54,680
to the SOP page and an action button to page now.

347
00:14:54,680 --> 00:14:59,000
Latency should be crisp, agentic retrieval with hybrid search and planar beats meandering

348
00:14:59,000 --> 00:15:00,080
chat.

349
00:15:00,080 --> 00:15:03,520
If it suggests contact the on call team, congratulations.

350
00:15:03,520 --> 00:15:05,480
You routed back to generic copilot.

351
00:15:05,480 --> 00:15:08,400
Check your capability tags and channel availability.

352
00:15:08,400 --> 00:15:10,040
Three, compliance.

353
00:15:10,040 --> 00:15:12,480
Can we email HIPAA updates with Outlook campaigns?

354
00:15:12,480 --> 00:15:13,480
Expect, no.

355
00:15:13,480 --> 00:15:14,800
Use patient portal with MFA?

356
00:15:14,800 --> 00:15:18,400
Bulk email prohibited with the legal memo citation and the service link.

357
00:15:18,400 --> 00:15:22,740
If you get a milk toast lecture on encryption, your index is wrong or your policy doc isn't

358
00:15:22,740 --> 00:15:24,240
authoritative in ranking.

359
00:15:24,240 --> 00:15:25,240
Tune re-ranking.

360
00:15:25,240 --> 00:15:27,880
Raise weight on compliance source of truth.

361
00:15:27,880 --> 00:15:28,880
Measure outcomes.

362
00:15:28,880 --> 00:15:32,680
Speed, compare default copilot's time to answer to the specialists.

363
00:15:32,680 --> 00:15:36,800
You should see fewer back and forths and lower end to end latency because the planar pulls

364
00:15:36,800 --> 00:15:38,200
exactly what's needed.

365
00:15:38,200 --> 00:15:41,360
Accuracy, sample answers for citations and correctness.

366
00:15:41,360 --> 00:15:42,760
Hallucinations.

367
00:15:42,760 --> 00:15:43,760
Track answers.

368
00:15:43,760 --> 00:15:46,680
Ranking citations and reduce them over time.

369
00:15:46,680 --> 00:15:50,240
Raise retrieval K, improve chunking at a disambiguation prompt.

370
00:15:50,240 --> 00:15:51,240
Itterate weekly.

371
00:15:51,240 --> 00:15:52,920
This is software not a statue.

372
00:15:52,920 --> 00:15:53,920
Final checklist.

373
00:15:53,920 --> 00:15:55,560
Schema 1.22 present.

374
00:15:55,560 --> 00:15:57,760
Copilot, agents and custom.

375
00:15:57,760 --> 00:15:59,400
Engine agents declared.

376
00:15:59,400 --> 00:16:01,000
Conversation status localized.

377
00:16:01,000 --> 00:16:02,480
Actions defined with parameters.

378
00:16:02,480 --> 00:16:03,960
Tenant controls configured.

379
00:16:03,960 --> 00:16:05,160
Data scopes enforced.

380
00:16:05,160 --> 00:16:06,560
RI filters active.

381
00:16:06,560 --> 00:16:07,560
Locks flowing.

382
00:16:07,560 --> 00:16:08,800
And three scenario tests.

383
00:16:08,800 --> 00:16:10,720
Passing with citations and links.

384
00:16:10,720 --> 00:16:12,720
Test that's green rollout beyond the pilot.

385
00:16:12,720 --> 00:16:14,480
Same UI, different brain.

386
00:16:14,480 --> 00:16:16,480
The intern just graduated.

387
00:16:16,480 --> 00:16:17,480
Proof?

388
00:16:17,480 --> 00:16:18,480
DLP policy.

389
00:16:18,480 --> 00:16:19,480
Regional SOPs.

390
00:16:19,480 --> 00:16:20,480
Compliance.

391
00:16:20,480 --> 00:16:21,480
Before versus after.

392
00:16:21,480 --> 00:16:22,840
Let's do the side by side.

393
00:16:22,840 --> 00:16:23,840
Same questions.

394
00:16:23,840 --> 00:16:25,160
Same users.

395
00:16:25,160 --> 00:16:26,920
Different brain.

396
00:16:26,920 --> 00:16:27,920
Case 1.

397
00:16:27,920 --> 00:16:29,240
DLP policy.

398
00:16:29,240 --> 00:16:30,800
Before default copilot.

399
00:16:30,800 --> 00:16:34,280
You ask, can I share a customer spreadsheet with an external vendor?

400
00:16:34,280 --> 00:16:38,680
It produces a smooth essay about safe sharing, labels and general DLP concepts.

401
00:16:38,680 --> 00:16:42,440
No mention of your exception list, no project Orion, no link to your runbook.

402
00:16:42,440 --> 00:16:47,120
The user nods, shares anyway, triggers an alert, and now security is babysitting cleanup.

403
00:16:47,120 --> 00:16:50,320
After the specialist, same prompt plus a project code.

404
00:16:50,320 --> 00:16:51,320
Response.

405
00:16:51,320 --> 00:16:55,520
No PII external sharing is blocked unless exemption project Orion is active.

406
00:16:55,520 --> 00:16:58,120
Current status inactive, submit exemption at requests.

407
00:16:58,120 --> 00:16:59,120
DLP exemption.

408
00:16:59,120 --> 00:17:00,120
Approval.

409
00:17:00,120 --> 00:17:01,120
Data protection office.

410
00:17:01,120 --> 00:17:02,120
C. DLP policy.

411
00:17:02,120 --> 00:17:03,640
Favore 3.2 and exception.

412
00:17:03,640 --> 00:17:04,640
Runbook.

413
00:17:04,640 --> 00:17:05,640
Favore 0.1.

414
00:17:05,640 --> 00:17:06,640
Two citations.

415
00:17:06,640 --> 00:17:07,640
One action.

416
00:17:07,640 --> 00:17:08,640
Zero poetry.

417
00:17:08,640 --> 00:17:10,680
Agent doesn't negotiate with ambiguity.

418
00:17:10,680 --> 00:17:15,000
It calls validate project code, checks the exception registry and answers with receipts.

419
00:17:15,000 --> 00:17:19,840
Accuracy improves because retrieval narrows to your indexed DLP artifacts and hallucinations

420
00:17:19,840 --> 00:17:23,120
plummet because the answer template requires citations to ship.

421
00:17:23,120 --> 00:17:24,120
Case 2.

422
00:17:24,120 --> 00:17:25,760
Regional SOPs after hours.

423
00:17:25,760 --> 00:17:26,760
Before default copilot.

424
00:17:26,760 --> 00:17:28,960
It's 1910 CET-72 in Emia.

425
00:17:28,960 --> 00:17:29,960
Who do I page?

426
00:17:29,960 --> 00:17:33,160
It responds with ITIL platitudes and contact the on-call team.

427
00:17:33,160 --> 00:17:34,160
Which one?

428
00:17:34,160 --> 00:17:35,160
How?

429
00:17:35,160 --> 00:17:36,160
Silence.

430
00:17:36,160 --> 00:17:37,160
Minutes pass.

431
00:17:37,160 --> 00:17:38,160
After.

432
00:17:38,160 --> 00:17:39,160
The specialist.

433
00:17:39,160 --> 00:17:42,920
It's 1910 CET-72 in Emia.

434
00:17:42,920 --> 00:17:43,920
Response.

435
00:17:43,920 --> 00:17:48,560
Page Emia vendor ops via Emia vendor SV2R-D.

436
00:17:48,560 --> 00:17:49,560
SLA.

437
00:17:49,560 --> 00:17:51,320
Acknowledge in 20 minutes.

438
00:17:51,320 --> 00:17:53,280
Escalate to duty manager if no arc.

439
00:17:53,280 --> 00:17:56,400
Use page now to send incident template.

440
00:17:56,400 --> 00:17:57,400
Citations.

441
00:17:57,400 --> 00:18:00,560
Emia SOP R2.3 and after hours playbook are for 1.1.

442
00:18:00,560 --> 00:18:01,560
The model didn't guess.

443
00:18:01,560 --> 00:18:02,560
It read your SOP.

444
00:18:02,560 --> 00:18:07,440
Use time and region as parameters and surface the exact channel plus the SLA.

445
00:18:07,440 --> 00:18:09,040
Speed 1 message to action.

446
00:18:09,040 --> 00:18:10,040
Accuracy.

447
00:18:10,040 --> 00:18:12,680
The name distribution list and escalation rule.

448
00:18:12,680 --> 00:18:13,840
Reduced hallucinations.

449
00:18:13,840 --> 00:18:17,040
No invented teams because the tool verified on call data.

450
00:18:17,040 --> 00:18:18,040
Case 3.

451
00:18:18,040 --> 00:18:19,040
HIPAA communications.

452
00:18:19,040 --> 00:18:20,640
Before default copilot.

453
00:18:20,640 --> 00:18:23,840
Can we email HIPAA updates using Outlook campaigns?

454
00:18:23,840 --> 00:18:25,240
It drifts into encryption.

455
00:18:25,240 --> 00:18:26,240
Data handling.

456
00:18:26,240 --> 00:18:28,080
Maybe suggest secure email.

457
00:18:28,080 --> 00:18:32,280
The average user interprets that as sure if careful as legal screams later.

458
00:18:32,280 --> 00:18:33,960
After the specialist.

459
00:18:33,960 --> 00:18:34,960
No.

460
00:18:34,960 --> 00:18:37,280
Patient notices require the patient portal with MFA.

461
00:18:37,280 --> 00:18:39,120
Bulk email is prohibited.

462
00:18:39,120 --> 00:18:42,760
Citations legal memo HIPAA comes our 5 and patient notice policy R2.

463
00:18:42,760 --> 00:18:43,760
Link.

464
00:18:43,760 --> 00:18:45,200
Open portal notice workflow.

465
00:18:45,200 --> 00:18:49,640
The agent calls the policy endpoint retrieves the governing memo and returns a crisp decision

466
00:18:49,640 --> 00:18:51,200
with a compliant path.

467
00:18:51,200 --> 00:18:52,200
Notice the pattern.

468
00:18:52,200 --> 00:18:53,920
Before is plausible narration.

469
00:18:53,920 --> 00:18:54,920
After is constrained.

470
00:18:54,920 --> 00:18:55,920
Decisioning.

471
00:18:55,920 --> 00:18:56,920
With sources and actions.

472
00:18:56,920 --> 00:19:00,920
That's why speed accuracy and fewer hallucinations show up together.

473
00:19:00,920 --> 00:19:04,560
It comes from a genetic retrieval plus tiny high signal tools.

474
00:19:04,560 --> 00:19:08,120
Accuracy comes from scoping to your indices and verifying with APIs.

475
00:19:08,120 --> 00:19:11,960
hallucinations drop because every final sentence has to stand on a citation.

476
00:19:11,960 --> 00:19:13,880
Let's quantify without the hand waving.

477
00:19:13,880 --> 00:19:14,880
Latency.

478
00:19:14,880 --> 00:19:18,680
Default copilot often needs 2 or 3 clarifying turns to stumble towards specificity.

479
00:19:18,680 --> 00:19:22,860
The specialist lands in 1 because the planner decomposes the prompt policy exception

480
00:19:22,860 --> 00:19:25,360
routing and fetches exactly those chunks.

481
00:19:25,360 --> 00:19:26,360
Token waste shrinks.

482
00:19:26,360 --> 00:19:28,400
The clock stops sooner.

483
00:19:28,400 --> 00:19:29,400
Detection rate.

484
00:19:29,400 --> 00:19:31,960
With default users correct the assistant or each other.

485
00:19:31,960 --> 00:19:35,400
With the specialist corrections become rare because the automation points to the single

486
00:19:35,400 --> 00:19:37,120
source of truth.

487
00:19:37,120 --> 00:19:38,520
Incident deflection.

488
00:19:38,520 --> 00:19:41,920
Support tickets about who do I page or is this allowed e?

489
00:19:41,920 --> 00:19:45,760
Decrease because the action buttons take people directly into the sanctioned workflow.

490
00:19:45,760 --> 00:19:47,840
Failure modes because reality exists.

491
00:19:47,840 --> 00:19:50,040
If after looks like before generic answers.

492
00:19:50,040 --> 00:19:52,560
You miss scoped data or forgot capability tags.

493
00:19:52,560 --> 00:19:54,640
So copilot never invoked your agent.

494
00:19:54,640 --> 00:19:58,200
If citations point to stale docs your index refresh is broken.

495
00:19:58,200 --> 00:20:01,600
If latency spikes your retriever is fishing too broadly.

496
00:20:01,600 --> 00:20:04,120
Tune hybrid search weights and re-ranker.

497
00:20:04,120 --> 00:20:07,560
And if a user still finds a hallucination, raise the citation threshold.

498
00:20:07,560 --> 00:20:09,280
No citation, no claim.

499
00:20:09,280 --> 00:20:10,280
Same UI.

500
00:20:10,280 --> 00:20:11,960
Different brain, that's the proof.

501
00:20:11,960 --> 00:20:14,480
Operations, life cycle, governance and scale.

502
00:20:14,480 --> 00:20:15,640
You build one specialist.

503
00:20:15,640 --> 00:20:17,760
Now act like an adult and run the program.

504
00:20:17,760 --> 00:20:19,160
Life cycle first.

505
00:20:19,160 --> 00:20:20,560
Version your agent like software.

506
00:20:20,560 --> 00:20:22,640
Vnext in dev, promotion to test.

507
00:20:22,640 --> 00:20:24,760
Then production via release tags.

508
00:20:24,760 --> 00:20:27,160
Componentize topics in action so you can reuse them.

509
00:20:27,160 --> 00:20:29,040
No copy-based policies.

510
00:20:29,040 --> 00:20:30,440
Automate evaluation.

511
00:20:30,440 --> 00:20:33,800
Regression prompts for DLP, SOP and compliance.

512
00:20:33,800 --> 00:20:36,560
Fail the build on citation drift or rising latency.

513
00:20:36,560 --> 00:20:39,160
Governance, scope identities with least privilege.

514
00:20:39,160 --> 00:20:42,160
Tie indices to data classifications and regions.

515
00:20:42,160 --> 00:20:44,600
EMEA agent doesn't see APAC records.

516
00:20:44,600 --> 00:20:47,760
Turn on audit, trace IDs, action logs retrieval sources.

517
00:20:47,760 --> 00:20:52,080
Security once at a stations, document models, actions, data flows and RAI filters.

518
00:20:52,080 --> 00:20:54,360
Publish risk notes for every new capability.

519
00:20:54,360 --> 00:20:56,440
Scale, treat adoption like product.

520
00:20:56,440 --> 00:20:58,720
Lead conversation status by persona.

521
00:20:58,720 --> 00:21:01,760
Monitor question coverage and add actions where user stall.

522
00:21:01,760 --> 00:21:07,520
Cost control, cap tool invocations, cash retrieval, right size models.

523
00:21:07,520 --> 00:21:12,440
And yes, set tenant controls by environment so experiments don't hit payroll.

524
00:21:12,440 --> 00:21:13,440
Key takeaway.

525
00:21:13,440 --> 00:21:18,280
Copilot becomes reliable only when your specialist agent brings retrieval, actions and

526
00:21:18,280 --> 00:21:20,320
guardrails into copilot chat.

527
00:21:20,320 --> 00:21:25,800
If this saved you time, repay the debt, subscribe, tap follow, enable notifications and

528
00:21:25,800 --> 00:21:29,240
watch the next deep dive on automated agent evaluation.

529
00:21:29,240 --> 00:21:31,120
Scheduled like a well-behaved con job.